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OFFLINE AND OFF-BUDGET: THE DISMAL 
STATE OF INFORMATION TECHNOLOGY 
PLANNING IN THE FEDERAL GOVERNMENT 


THURSDAY, JULY 31, 2008 

U.S. Senate, 

Subcommittee on Federal Financial Management, 
Government Information, Federal Service, 
and International Security, 
of the Committee on Homeland Security 
and Governmental Affairs, 
Washington, DC. 

The Subcommittee met, pursuant to notice, at 9:33 a.m., in room 
SD-342, Dirksen Senate Office Building, Hon. Thomas R. Carper, 
Chairman of the Subcommittee, presiding. 

Present: Senator Carper. 

OPENING STATEMENT OF SENATOR CARPER 

Senator Carper. The Subcommittee will come to order. Our 
thanks to our guests and witnesses for being here with us today. 
This is the third hearing that our Subcommittee has held on the 
issue of poorly planned and poorly performing IT investments. This 
hearing will focus once again on the ability of the Office of Manage- 
ment and Budget to oversee and provide Congress with visibility 
into the $70-plus billion that agencies will spend on information 
technology this year. 

Up until March of this year, Congress had an extremely limited 
ability to understand why OMB considered an IT investment to be 
poorly planned, and I must commend OMB and in particular Ms. 
Evans for finally releasing this data. Although I believe this is a 
good start, more complete and accurate information needs to be 
shared. I firmly believe that in order to hold agencies accountable 
for their investments, Congress and OMB need to work together as 
partners. The American taxpayers demand it. 

Information technology investments, if planned and implemented 
properly, can increase productivity, improve efficiency, and reduce 
an agency’s cost, and also enable us to provide better service to our 
constituents. However, some of these projects can be extremely dif- 
ficult to manage and mistakes may be made along the way. 

In fact, I think it was Richard Nixon who said the only people 
who don’t make mistakes are the people who don’t do anything. So 
we know if we try to do something in these complex areas, we are 
going to make mistakes. That is to be expected. The key is, I think, 
to make sure that we don’t continue to make the same mistake 
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over and over again. However, the mistakes that we do make, we 
learn from them. 

I experienced this firsthand when I was privileged to serve as 
Governor of Delaware. Sometimes we bit off more than we could 
chew and the IT project would eventually spiral out of control. 
When this happened, sometimes we came to the conclusion that the 
best course of action was just to finally pull the plug. It was a 
tough decision a lot of times, but it was often the right thing to do 
in certain cases. 

Unfortunately, many agencies in the Federal Government are al- 
lowed to spend billions of taxpayer dollars on investments that are 
duplicative, that lack clear goals, and that are managed by un- 
qualified individuals. In fact, according to recently-released GAO 
data, some $25 billion in IT investments are poorly planned, poorly 
performing, or both. Even worse, some of these projects have been 
delayed up to a decade and are costing us billions more than was 
originally expected. This is simply unacceptable and it makes me 
wonder whether it is time for Congress to pull the plug on some 
of these failed investments. 

Regrettably, Congress still does not have the information nec- 
essary from OMB to hold agencies accountable and to choose where 
we want to invest scarce resources each year. With risky invest- 
ments such as IT, it is important to increase collaboration and visi- 
bility, not hinder it. And so far, this has not always been the case. 

Since 1994, Congress through legislation called the Federal Ac- 
quisition Streamlining Act has required agencies to keep costs, de- 
livery dates, and performance goals of major acquisition within 90 
percent of the originally proposed plan. OMB then was required to 
annually report to Congress on agency progress. 

Unfortunately, despite this requirement in law, OMB has, I am 
told, only issued three reports in the last 14 years. Although my 
staff tells me we may have received a report around midnight last 
night. Someone here was staying up late and I think may have re- 
ceived a fourth report in 14 years in the midnight hour. 

Moreover, agencies are required to create an investment baseline 
that takes into account potential risks that could lead to increased 
costs, delayed delivery dates, and reduced performance. Agencies 
then use this baseline to track whether an investment is pro- 
gressing according to plan. 

There are obviously some legitimate reasons why an agency may 
change the original baseline on a given project, but I am dis- 
appointed to say that some agencies have used rebaselining to hide 
the cost overruns or schedule delays from Congress. 

We are about to hear an extremely troubling report from the 
Government Accountability Office today revealing that almost 50 
percent of all Federal agency IT investments are rebaselined. Even 
more disturbing, some agencies, such as the Department of Agri- 
culture, the Department of Commerce, Veterans Affairs, have 
rebaselined, I understand, more than five times on a single invest- 
ment. 

Although agencies are responsible for the excessive rebaselining, 
there is one thing in common between all of these investments. 
Every baseline and rebaseline was approved by OMB. Someone, 
somewhere, in my view, is not fulfilling their responsibility to en- 
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sure the taxpayer dollars are spent only on those investments that 
are well thought out and truly needed. 

Using the information that was provided to our Subcommittee in 
March, we have created a report card for agencies that take into 
account several criteria related to the planning and implementa- 
tion of IT investments. In fact, we have two report cards side by 
side. I say this as a parent whose children have both now grad- 
uated from high school and who is used to getting report cards. 
Sometimes my children aren’t anxious for us to see these report 
cards. But on the report card on the left, we have a list of agencies 
who received a passing grade, and on the right, those who receive 
a failing grade. If anybody in the audience is able to read those, 
I take my hat off to you. We have copies of the report cards that 
are provided to our witnesses and I hope to the press. 

As we can clearly see the Federal Government is miserably fail- 
ing. In fact, half the 28 agencies received an F. Let me go ahead 
and list some agency grades starting with the Department of 
Labor, whose score was about 64 percent. The Department of 
Treasury, 56 percent. Office of Personnel Management, 55 percent, 
all the way to the bottom of this chart on the right, 1 down to the 
Department of Defense with 37 percent, followed by the Depart- 
ment of Agriculture with a 36 percent score. 

About half of the 28 agencies received an F. In total, these failing 
agencies are overseeing $57 billion in IT investments, and from 
what we can tell right now, we are getting too little in return for 
those enormous investments. 

In addition, you might notice on the left, we have a number of 
agencies who receive a passing grade. Several received scores of A- 
plus. They included HUD, the National Science Foundation, the 
Smithsonian Institute — they all got 100 percent — the Social Secu- 
rity Administration, 99 percent, Department of Energy, 93 percent, 
all the way down to the bottom of the chart, coming in at a 69 per- 
cent for a D-plus, and that is the Department of Justice. 

However, and while we are encouraged by the passing grades in 
the report card on the left, we want to temper our joy by a couple 
of things. HUD received their A-plus partly because they reported 
that every project was being delivered on cost, on schedule, and 
performing as planned. However, as GAO will testify today, HUD 
rebaselined at least one project seven times, possibly to mask spi- 
raling costs, and GAO has consistently testified that data provided 
by agencies to OMB is oftentimes inaccurate or even incomplete. 

So we don’t really have the complete picture, even for those agen- 
cies that are reportedly doing a better job than others. That is why 
I plan on introducing legislation today, along with Senator Lieber- 
man, Chairman of our full Committee, and Senator Collins, the 
Ranking Republican on the full Committee, which will give Con- 
gress and OMB the information we need to make better decisions 
about which IT investments should continue and which should be 
shut down. 

Our bill, called the Information Technology Investment Oversight 
Enhancement and Waste Prevention Act, would make agencies re- 
port regularly on significant deviations on cost, schedule, and per- 


lr The chart referred to appears in the Appendix on page 135. 
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formance. But we don’t just want better information. Our bill also 
helps OMB take a crucial step aimed at preventing IT investments 
from drifting toward failure. Recognizing that agencies may not 
have the skills necessary to manage complex IT investments and 
may have trouble recruiting qualified managers, our bill would set 
up a team of experts from inside and outside of government that 
agencies may use as a resource. This team, which my staff and I 
regard as something of an IT strike force, would have the skills 
and background necessary to make sure that agencies are focused 
on the right things, making the right decisions, and spending the 
money wisely. 

I again want to thank our witnesses for joining us today. We look 
forward to hearing from you. We look forward to your testimony. 
We look forward to the discussion that will follow it and hopefully 
to the better IT performance, for the money that our taxpayers are 
investing in those projects today. Taxpayers expect us to be good 
stewards of their money. In fact, they demand it, and I know that 
everyone here in this room wants to see that become a reality. 

Before I introduce Mr. Denett, Ms. Evans, and Mr. Powner, none 
of whom are strangers to this Subcommittee or to this room, I real- 
ize these are tough issues. These are not easily done. If they were, 
somebody would have done them a long time ago. The problems 
were recognized not just in this Administration, but in the last Ad- 
ministration, as well. While we appreciate that the problems are 
being identified, what is really important is that we identify them 
more quickly and that we fix them. I believe that we need good in- 
formation here in the Legislative Branch so that we can be better 
partners with the Executive Branch. 

We are going to see a new Administration coming in in 6 months. 
It is going to be President McCain or President Obama. But these 
problems and these challenges will still be before the next Adminis- 
tration and the next Congress. 

As Governor, I used to say the reason why we invested money 
in these IT projects was to enable us to provide better constituent 
service and to do it more cost effectively, and that is really the case 
here, too. In some areas, we are doing a pretty good job. In too 
many areas, though, we are not. In too many areas, we are mask- 
ing mismanagement, misallocation of resources and cost overruns 
in ways that make it look like we are doing a better job on some 
projects than we truly are. We have to get beyond that. 

With that having been said, on that cheery note, we have a cou- 
ple of panels before us. In the interest of time, I am just going to 
move right to the first panel for their testimony. Biographies of our 
witnesses are provided and will be submitted for the record. 

Our first witness is the Hon. Paul Denett. Mr. Denett serves as 
the Administrator of the Office of Federal Procurement Policy at 
the Office of Management and Budget. Previously, Mr. Denett 
served as counselor to Clay Johnson, who is now the Deputy Direc- 
tor for Management at OMB. Mr. Denett is a retired Senior Execu- 
tive from the Federal service and has received many prestigious 
awards, including a Presidential Rank Award. When did you re- 
ceive that? 

Mr. Denett. I received that in the late 1990s. 

Senator Carper. Did you really? Who was President then? 
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Mr. Denett. President Clinton. 

Mr. Denett. Yes, it was. 

Senator Carper. All right. Some of us are lost in the 1960s. 
[Laughter.] 

Some days, I would like to be, but not today. [Laughter.] 

All right. Our next witness is the Hon. Karen Evans. Ms. Evans, 
welcome. She is Administrator of the Office of Electronic Govern- 
ment and Information Technology at OMB. Ms. Evans oversees the 
implementation of IT throughout the Federal Government, includ- 
ing capital planning and investment control. She is a 20-year vet- 
eran of government service and has testified before this Sub- 
committee on a number of occasions. It is a pleasure to have you 
join us again today. Thank you. 

Our final witness is Dave Powner, Director of Information Tech- 
nology Management at the Government Accountability Office. I am 
glad that Mr. Powner doesn’t charge us for each visit that he 
makes and each time he testifies before this Subcommittee, be- 
cause we would be owing him a lot. But in the private sector, he 
has held several executive level positions. In the telecommuni- 
cations industry, Mr. Powner has been instrumental in helping this 
Subcommittee provide oversight of risky IT investments, along 
with other things. I know he has appeared before us on several oc- 
casions, including as we try to address our current challenges in- 
volving conducting a good Census for the year 2010. 

We are going to lead off and ask Mr. Denett to lead off. We will 
ask you to keep your statements to roughly 5 minutes. If you go 
a little bit beyond that, that is OK. If you go a lot beyond that, that 
is not OK. So we will turn to you now. As you present your testi- 
mony, just keep in mind that your full statements will be entered 
into the record. 

TESTIMONY OF HON. PAUL A. DENETT , 1 ADMINISTRATOR, OF- 
FICE OF FEDERAL PROCUREMENT POLICY, U.S. OFFICE OF 

MANAGEMENT AND BUDGET 

Mr. Denett. Thank you. Chairman Carper, Ranking Member 
Coburn, and Members of the Subcommittee, I appreciate the oppor- 
tunity to appear before you today. The Administration places a 
high priority on working with agencies to mitigate cost overruns, 
schedule delays, and performance shortfalls in their major acquisi- 
tions. 

This morning, my colleague, Karen Evans, will discuss the initia- 
tives the Administration is pursuing to effectively manage the gov- 
ernment’s IT portfolio. I would like to briefly summarize agencies’ 
progress in implementing performance-based management for their 
major non-IT acquisition programs. I have also prepared written 
remarks that I ask the Subcommittee to enter into the record. 

Similar to IT, major acquisitions of non-IT capital assets must be 
justified in terms of agencies’ strategic goals and reflect sound ac- 
quisition and capital planning decisions. The law requires agencies 
to apply performance-based management principles by establishing 
cost, schedule, and performance goals and achieving 90 percent of 
these goals, on average. 


1 The prepared statement of Mr. Denett appears in the Appendix on page 39. 
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OMB guidance for meeting these requirements is set forth in Cir- 
cular A— 11, which provides guidance to agencies on preparing their 
budget submissions, and the Capital Programming Guide, which is 
a supplement to A-ll. The guide was substantially revised in 2006 
to emphasize the importance of key steps in the acquisition plan- 
ning process, such as needs assessment and alternative analysis. 

To assess the use of performance-based management, my office 
directed agencies to provide information on their new and ongoing 
non-IT major acquisition projects. We look for several indicators of 
progress, including the existence of capital planning and invest- 
ment control policies, cost schedule and performance goals for new 
and ongoing projects, use of performance-based management sys- 
tems to monitor progress and success in meeting goals. 

Results were mixed. While many agencies demonstrated prog- 
ress, we found that capital planning policies for non-IT are often 
not as well established as they are for IT investments. In addition, 
performance-based management systems are not always being used 
to track cost schedule and performance. We can and must do bet- 
ter. 

A number of steps are being taken to strengthen the application 
of performance-based management to non-IT investments. First, 
the Chief Acquisition Officers’ Council created a Project Manage- 
ment Working Group to help OMB evaluate the appropriate appli- 
cation of performance-based management to different types of 
major non-IT investments, such as construction and aircraft. 

Second, OMB and GAO are partnering with DOD, DOE, and 
NASA, each of whom is on the GAO’s High-Risk List, in the devel- 
opment and implementation of corrective action plans to mitigate 
risk in major acquisitions and evaluate success against clear goals 
and metrics. 

Third, the Office of Federal Procurement Policy and the Federal 
Acquisition Institute launched a Federal Acquisition Certification 
Program to provide civilian program and project managers with 
standardized training on competencies critical to successful per- 
formance-based management. These include requirements analysis, 
cost estimating financial management, risk management, and qual- 
ity assurance. Program and project managers that are assigned to 
major acquisitions will need to be certified under this program. 

Finally, OFPP created a standardized approach for agency self- 
assessments of the acquisition function based on a framework that 
was created by GAO. Self-assessments of major acquisition func- 
tions will address issues to help agency managers understand if 
planning is effective. If a material weakness is identified, it will be 
addressed during corrective action monitoring and reflecting on the 
agency’s statement of assurance prepared under OMB Circular A- 
123, which lays out management’s responsibilities for internal con- 
trols. 

In accordance with the Federal Acquisition Streamlining Act, 
OFPP issued a report — I think it was a little before midnight, but 
it was late — discussing civilian agency progress in implementing 
performance-based management 

Senator Carper. Actually, I am told it was 12:04. [Laughter.] 

But who is keeping time. [Laughter.] 
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Mr. Denett. OK. For both IT and non-IT investments. We will 
be happy to answer any follow-up questions the Subcommittee may 
have. Thank you. 

Senator Carper. Right on the money. Thank you. Ms. Evans, 
welcome. 

TESTIMONY OF KAREN S. EVANS , 1 ADMINISTRATOR, OFFICE 

OF ELECTRONIC GOVERNMENT AND INFORMATION TECH- 
NOLOGY, U.S. OFFICE OF MANAGEMENT AND BUDGET 

Ms. Evans. Good morning, Mr. Chairman. My remarks will focus 
on the Administration’s strategy and continued progress in per- 
forming oversight of agencies’ capital investments and information 
technology. Specifically, I will address the topic of OMB’s ability to 
effectively analyze, track, and evaluate agencies’ major capital IT 
investments. 

The President’s budget request for fiscal year 2009 includes ap- 
proximately $71 billion for IT and associated support services gov- 
ernment-wide. Of the $71 billion in the President’s budget request, 
$22 billion, or 31 percent, represents proposed funding for develop- 
ment, modernization, or enhancement of new or existing informa- 
tion systems, infrastructure, or services. Projected expenditures on 
existing systems, operations, and maintenance is $49 billion, or 69 
percent. 

As I have discussed in previous testimony on the topic of OMB’s 
oversight of IT investments, the Clinger-Cohen Act of 1996 estab- 
lished processes for executive agencies to analyze, track, and evalu- 
ate the risks and results of major capital investments for informa- 
tion systems. The operative means by which OMB evaluates agency 
capital asset plans and their associated budgetary requests is a 
business case, or Exhibit 300, for individual investments. 

Agencies develop and submit business cases with their annual 
budget requests to OMB and we, in turn, evaluate each business 
case in terms of its ability to support a given investment proposal, 
including factors such as alignment with IT architecture, plan per- 
formance, improvement goals, cost-benefit analysis, and elimi- 
nating costly, duplicative, and outdated systems. OMB also evalu- 
ates the capability to manage the investment as demonstrated in 
the business case and planning process, including factors such as 
having a qualified project manager, acquisition planning, systems 
security, risk management, and the use of earned value manage- 
ment to track and manage costs and schedule goals. 

Last year, I came before this Subcommittee to explain two spe- 
cific tools OMB uses to track and review agency IT capital invest- 
ments, the Management Watch List and the High-Risk List. I am 
pleased to report that we have since expanded upon our criteria for 
evaluating investments and improved upon the transparency of 
how investments and projects are placed on either or both lists. My 
written statement includes the details regarding these lists. 

GAO’s report on project rebaselining points out weaknesses in 
agencies’ policies in terms of specifying all of the elements of re- 
baselining according to best practices. We acknowledge more 
should be done in this area in the future. In particular, OMB needs 


lr The prepared statement of Ms. Evans appears in the Appendix on page 45. 
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to clarify expectations for when an original baseline should be es- 
tablished and elaborate upon process steps for agencies to submit 
to and receive feedback back from OMB on rebaseline requests. 

OMB, Congress, and the agencies must work collaboratively to 
address weaknesses in IT program and project performance. We 
are all vested in a common interest, delivering results for the 
American people. 

How can we best do this going forward? I believe OMB has the 
foundational processes in place to perform program and project in- 
vestment oversight at the macro level. These core processes — cap- 
ital planning, architecture alignment, the Management Watch List, 
the High-Risk List, and the E-GOV scorecard — have been and can 
continue to be further enhanced and incrementally improved to 
provide better visibility into program performance before a project 
is in major trouble. 

One improvement for OMB would be to leverage the efforts of the 
budget formulation and execution line of business to ensure the use 
of analytical tools and the collaboration environment to improve on 
our own information management capabilities. The various data 
sets collected by OMB from the agencies can and should be better 
integrated into a more comprehensive knowledge base. 

I have brought a display of an example of how we could better 
integrate and expand upon information on the agency’s IT invest- 
ments. This is one potential snapshot of project performance at a 
point in time. We would like to work with Congress to improve 
transparency and the ease of use of this information. With the 
transparency into departments’ and agencies’ performance will 
come improved accountability and results which the American peo- 
ple deserve. 

Thank you for this opportunity to discuss the Administration’s 
strategy and progress to date. With this foundation in place, we 
can continue to work together to achieve the outcome we both de- 
sire, which is successful implementation of information technology 
for program and mission results. We have accomplished a lot in the 
last 8 years and there is much more that we can continue to do 
in this area. 

I would be happy to take questions at the appropriate time. 

Senator Carper. Thank you for that testimony. Mr. Powner. 

TESTIMONY OF DAVID A. POWNER , 1 DIRECTOR, INFORMATION 

TECHNOLOGY MANAGEMENT, U.S. GOVERNMENT ACCOUNT- 
ABILITY OFFICE 

Mr. Powner. Chairman Carper, we appreciate the opportunity to 
testify this morning on poorly planned and performing IT projects 
and the results of our rebaselining review completed at your re- 
quest. 

Mr. Chairman, I would like to thank you and Dr. Coburn for 
your oversight of the Federal IT budget, which now exceeds $70 bil- 
lion. This is the third annual hearing, and your oversight, along 
with the leadership of Karen Evans, has resulted in a clear picture 
of exactly what projects need work and the specific weaknesses 
that need to be addressed. That is the good news. The bad news 


x The prepared statement of Mr. Powner appears in the Appendix on page 50. 
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is that we have nearly $25 billion in IT investments that are cur- 
rently at risk of being wasted and that this figure is likely much 
higher. 

This morning, I have three points to make. First, over 400 IT 
projects totaling nearly $25 billion are currently not appropriately 
planned for or managed. 

Second, the number of projects reporting issues with cost and 
schedule variances is understated because rebaselining is occurring 
excessively without adequate guidance or transparency. 

And third, additional oversight from both OMB and agency CIOs 
is needed to address project weaknesses and to bolster account- 
ability. 

Expanding on each of these, first, currently, there are about 350 
projects totaling $23 billion on the Management Watch List, and 
nearly 90 projects totaling $5 billion that are being reported as 
high-risk projects with shortfalls. Common to both lists are 26 
projects totaling about $3 billion, meaning that these projects are 
both poorly planned and performing. For example, DHS’s Secure 
Border Initiative project continues to be on both lists. 

In addition to the 26 projects on both lists, of particular concern 
are the repeat offenders, namely 32 projects that have been on the 
Management Watch List since September 2006 and 17 high-risk 
projects reporting shortfalls in each of the last four quarters. My 
written statement highlights each of the projects on both lists and 
those that have been on either list far too long. 

Mr. Chairman, having so many projects with longstanding plan- 
ning and performance problems is unacceptable and requires more 
attention from both OMB and agency CIOs. Key reasons why 
projects remain on the Management Watch List are poor cost and 
schedule performance, poor security measures, poor privacy plan- 
ning, and poor project management, while cost and schedule 
variances continues to be the primary performance shortfall associ- 
ated with high-risk projects. 

Despite cost and schedule performance being a major reason why 
so many projects are highlighted here, our rebaselining report 
being released today raises significant questions about whether we 
are getting an accurate picture of project cost and schedule per- 
formance. 

First, I would like to mention that rebaselining is needed at 
times to reset realistic cost and schedule targets. However, this 
process should not be used to mask cost and schedule overruns and 
should be transparent and approved by both OMB and agency 
management. Our survey of nearly 200 projects showed that about 
half of all IT projects had been rebaselined, and of those that are 
rebaselined, about 50 percent are rebaselined at least twice, and 10 
percent are rebaselined four or more times. Clearly, these projects 
are operating nowhere near the desired 10 percent cost and sched- 
ule threshold. 

Because of the amount of rebaselining occurring across the Fed- 
eral Government, we reviewed each of the major 24 departments’ 
rebaselining policies to ensure that it included basic items like 
specifying when a rebaselining is warranted, how a new rebaseline 
is validated, and who approves the new rebaseline. Agency policies 
were weak across the board and we recommended that OMB issue 
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rebaselining guidance and that each agency develop rebaselining 
policies that address the weaknesses we identified. 

Although we have more data than we have ever had historically, 
we still need better information so that all projects requiring atten- 
tion are highlighted. For example, OMB could publicize high-risk 
projects with shortfalls as we have in our written statement. And 
we still need more transparency on rebaselining efforts so that we 
have a true picture of cost and schedule performance from all agen- 
cies. 

Put more simply, Mr. Chairman, do we think that of the 472 
high-risk projects only 70, or 15 percent, have cost and schedule 
issues? No. That number should clearly be higher. 

Now turning toward solutions, the Federal Government needs to 
focus on addressing root cause problems: Security, project manage- 
ment, risk management, and cost and schedule performance. We 
are encouraged by OMB’s efforts to address some of these weak- 
nesses, like requiring techniques like earned value management to 
improve agencies’ cost and schedule performance, but there is 
much more work ahead. Agency CIOs and OMB from a govern- 
ment-wide view need to aggressively attack these problem projects 
by starting with those that have been on OMB’s radar far too long. 

Next, longer-term improvement efforts need to be pursued and 
these, at a minimum, need to focus on bolstering the IT workforce 
and addressing root causes more aggressively. I look forward to 
suggestions from our second panel of experts on other approaches. 

In summary, Mr. Chairman and Dr. Coburn, knowing what to fix 
is the first step, and we wouldn’t have this information without 
your leadership. Thank you. Keeping the current momentum and 
energy and focusing more on solutions will be essential as we tran- 
sition to a new Administration. 

I would be pleased to respond to questions. 

Senator Carper. Good. Thank you for that testimony. Thanks 
very much for the work that preceded it. On behalf of both Senator 
Coburn and myself and our Subcommittee and staff, thank you for 
the very good work that you have done. 

What I hope we focus on more today — yes, the sun is coming up. 
I don’t know if it seemed dark to the rest of you, but it seemed 
dark to me. I just asked my staff to brighten the lights. So if you 
are not awake, wake up now. [Laughter.] 

I was wondering, who are some good models on the public or pri- 
vate sector for the Federal Government? Who out there is man- 
aging IT projects well? Not just in planning and developing them, 
but implementing them on time and fairly close to budget without 
rebaselining six or seven times on a single project? Do we have any 
idea who is doing an especially good job and why? 

Mr. Powner. I can tell you from our focus on some of the high- 
risk modernization efforts that we look at, even though some of 
these organizations received low department grades, if you look at 
pockets within the Federal Aviation Administration, if you look at 
IRS, recently there have been successes with delivering more 
projects on time and within budget. And if you look at what has 
resulted, there has been an extreme focus on improving their proc- 
esses, executive governance over those projects, and one other tech- 
nique that seems to have worked at some of those agencies is put- 
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ting in place Centers of Excellence where project offices can go for 
help, where there is best practices and things they can emulate 
across the board. They are not perfect by any means, but there are 
pockets of success in both of those agencies. 

Senator Carper. Thank you for that response. In the National 
Governors Association, all 50 governors are members of that, we 
had a Center for Best Practices and if they had a particular State 
that was doing a good job in early childhood, the rest of us could 
learn from them. If we had a particular State that was doing a 
good job in reducing recidivism in the prison population, or if a 
State was doing an especially good job in providing better outcomes 
in our schools, we had the opportunity through the Center for Best 
Practices to see who was doing a good job and learn from them. Do 
we have that kind of ability in the Federal Government? 

Ms. Evans. Yes, sir. Through the CIO Council, we have a Best 
Practices Committee where we do take on topics such as these spe- 
cific areas, like who has implemented earned value management 
really well and can people learn from that. We need to do more in 
those Best Practices Committees, and I would also like to build off 
of what Mr. Powner talked about. 

What we have been doing from an OMB perspective is really 
zeroing in in particular areas. The two areas that Mr. Powner men- 
tioned are also on the GAO High-Risk List. So there is a lot of 
work that we have done in conjunction with the agency, in conjunc- 
tion with GAO, so that we could then cross-pollinate the expecta- 
tions, the best practices in those areas, and then the intent is to 
take that and then share it back across the board with other agen- 
cies. 

So we have done that in the area of, with NARA and the Elec- 
tronic Archives Initiative. As they go forward, what we have 
learned from FBI on their Sentinel project is they have improved. 
We have then partnered them up with other agencies as they are 
going forward on major investments so that they make sure that 
they don’t make the same mistakes. 

Senator Carper. Mr. Denett. 

Mr. Denett. We also have the Chief Acquisition Officers Council, 
and this past year to try to respond to the problems in the program 
project management area formed a new working group. It is 
chaired by Bill McNally, who is the Senior Procurement Executive 
of NASA. But all the major departments send representatives and 
they are sharing best practices and we are going to post them on 
the Federal Acquisition Institute website. 

Senator Carper. All right. Mr. Powner. 

Mr. Powner. I would just like to add that I agree with the com- 
ments that were made here, but one important thing that I think 
Ms. Evans made, an important comment, is the implementation of 
those best practices. Earned value management, a very good tech- 
nique when used appropriately. Our preliminary work on earned 
value management implementation across the government is that 
it is rather immature, and I will give you an example. 

We just completed a review of FAA. It is likely one of the better 
agencies when it comes to implementing earned value. There has 
been a lot of focus and attention. But I can tell you there that a 
quarter of their projects don’t use earned value. Not everyone is 
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trained. And when you start looking at the data, the earned value 
data — and we did, we dove deep on a couple projects — we saw quite 
a few problems that we pointed out. 

Now, granted, that is probably one of the better Federal agencies, 
but when we look at the cost and schedule problem that we have 
in the Federal Government coupled with rebaselining, we have to 
get much better at using these techniques like earned value to 
make good management decisions to get early warning indications. 

Senator Carper. All right. Thank you. The next question I want 
to ask, and I have touched on this already, but we started this 
hearing off with a little broader context than I think we had origi- 
nally planned to discuss, but it is something that I am troubled by 
and I just want to come back to it again and discuss it. 

One of the Subcommittee’s purposes is, as you know, to prevent 
waste, fraud, and abuse. In order to do that, we need information 
on how agencies are managing our investments. Mr. Denett, in 
1994, Congress directed your office to provide a report on agency 
progress in delivering on their capital investments, including IT in- 
vestments. However, as I stated in my opening statement, in 14 
years, OMB has only provided this report on three separate occa- 
sions, now maybe four. Let me just ask, why is OMB not fulfilling 
this critically important statutory requirement? 

Mr. Denett. It was a mistake on our part, for sure. When I be- 
came aware of this, we have had the staff work diligently, go out 
to departments and pull it in, and I don’t have a good explanation 
as to why some years were not submitted. However, there is the 
annual budget process under A-ll where a lot of the data that is 
included in the report is submitted with the President’s budget 
each year, so that contains some of the information we sought. But 
it should have been wrapped up, put into a formal report, and pro- 
vided to you each year. We have now put steps in to make sure 
that this doesn’t happen in future years. 

Senator Carper. What can you do? Who is responsible for mak- 
ing sure this statutory requirement is met? 

Mr. Denett. The Office of Management and Budget, so we are. 

Senator Carper. All right. What assurance can you provide for 
us that it is going to be met in the future? 

Mr. Denett. We have added it to a follow-up list that OMB has 
so that the new Administration will have it as a reminder before 
them. We have told the staff the importance of it, and it is not our 
prerogative to decide what years to submit it and what not, that 
we all need it as a tracking tool. This one will be stronger. We are 
going to have our own baselines on which ones are — just as you are 
having your report card, we are going to have that kind of method 
in there, and I believe in the future, you will get the reports each 
year. 

Senator Carper. All right. I certainly hope so. 

Ms. Evans, this is the third hearing during which you have testi- 
fied how billions of taxpayer dollars are wasted on bad IT invest- 
ments. I want to commend you for all the progress that you and 
your team have made over the past several years. However, I be- 
lieve that there is still more that needs to be done. I am sure you 
do, too. In fact, you said that. 
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Specifically, in 2006, GAO first recommended that OMB ensure 
agencies are accurately and reliably reporting on their investments. 
Also, GAO recommended that OMB provide Congress a single ag- 
gregate list for troubled IT investments that can allow us to track 
progress and performance government-wide, and it is now 2 years 
later. 

Just revisit with us, if you will, what specific progress since 2006 
have you and your colleagues made toward ensuring that agencies 
are reporting complete and accurate information to OMB. And sec- 
ond, why doesn’t OMB publish the list of high-risk projects with 
shortfalls as GAO highlights in their testimony? I am just asking, 
don’t you feel that Congress would want to know whether projects 
are over-budget and behind schedule? 

Ms. Evans. Sir, I actually would like to answer the first question 
first 

Senator Carper. Please. 

Ms. Evans [continuing]. Which would be what we have done to 
increase the transparency in this. And I would say, first off, with 
the hearing and everything and the several hearings along this 
line, that I was hesitant at first, which I think you would admit, 
to releasing all of this information because we were concerned 
about some of the effects that it would have. 

Senator Carper. What do you mean 

Ms. Evans. I was going to go into this a little bit. And some of 
it, you are talking about and we have kind of gone around it a little 
bit, which is driving compliance versus actually achieving the re- 
sults. And so things happen like the grades, not that this is not 
great because OMB has an A, so I am pleased that we have an A. 
But it is an A-minus. I would have preferred an A-plus, but I know 
why it is a minus. But what this will do is drive 

Senator Carper. I asked my staff if we were grading on a curve 
here. They said, no, we are not. 

Ms. Evans. No, but I don’t have as many investments as every- 
one else, so 

Senator Carper. We are calling them as we see them. 

Ms. Evans. But part of the issue is, one, that there is a part of 
us that love to see the grades, and so we strive for this. This is how 
our society is built. But the other part then drives compliance, 
where people will do just enough in order to get to the next level 
of the grade and they are focused more on the grade than actually 
achieving the result or getting the project implemented. 

And so this is a balance that my office has really been struggling 
with, about how much information do we release, how much shame 
and embarrassment do you bring upon an agency, because we are 
really supposed to be helping them. We are supposed to be there 
to help the agency achieve success. We are supposed to take areas 
in our knowledge that we see across the government and go in and 
help an agency achieve results. 

So I agree that all this information should be out there. I don’t 
necessarily disagree that we could provide better reporting, not 
necessarily along the lines of GAO, because I don’t view OMB as 
an auditor. I view OMB as a facilitator to help the Executive 
Branch achieve results. 
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So I would like us to be able to provide the information in a way 
that you would find it useful as an oversight entity the same way 
that we find it useful so that we could get to the solution for the 
agency and for the taxpayers. 

So we have done a lot to put all this out here. We have not put 
together the comprehensive list the way that GAO does or the way 
that it was written. I brought two examples, which are also in the 
written testimony, and if we want later on, I could go into a couple 
areas of how it could be done. We put a lot of effort into this so 
that we could match and cross-reference and make sure that all of 
this is reliable data, and then we could work with you in a way 
similar to the way that we have done with USA Spending and even 
release some of the raw data so that as you go forward, you guys 
could manipulate and do things with the data, as well. 

So I am always very cautious as we go forward on this because 
I am always really cautious about driving compliance versus trying 
to get to the result. 

Senator Carper. All right. A related question. You may have spo- 
ken to this, but is there a reason why OMB hasn’t provided Con- 
gress with a single aggregate list that will allow us to provide prop- 
er oversight and track progress? Just try that. I think you have an- 
swered that question, but I want you to take another shot at it. 

Ms. Evans. I was hoping you didn’t realize I didn’t answer that 
part [Laughter.] 

But I will tell you, the reason why, and this is our first — the ta- 
bles that you have are our first attempt at aggregating the data to- 
gether. The short answer is that based on what we had previously 
to provide the aggregated report, we believe would cause more con- 
fusion than actually bring clarity to the result. And then the other 
piece was we had to work a little bit more with the agencies to get 
better quality data in order to establish the relationship to do an 
aggregate report. I believe we are there now and what we would 
like to work with you on is how to actually provide that informa- 
tion in a way that would be useful for your use. 

Senator Carper. Why not just provide the same information as 
GAO? 

Ms. Evans. Because I would say that right now, we have several 
pieces of that, but we don’t go in and do full audits the same way 
that GAO does. So some of the information, for example, like the 
number of times that a project has been rebaselined, we would 
have to modify some of our A-ll guidance. Right now, we just cap- 
ture that information as a yes and no, and I have not captured that 
information on an ongoing basis so that I could go down and map 
all the records and say that this investment has been rebaselined 
several times, X, Y, Z, and this year, those types of things like that. 

I have not kept the information at the micro level. The agencies 
are supposed to keep it at that level. And what we are trying to 
do is make sure that there is better quality as we start putting it 
forward. So I would be hesitant to publish the information because 
I do not feel that we have the quality still at that real detailed 
level because we haven’t conducted the audit as GAO has. 

Senator Carper. I am going to ask Mr. Powner to comment on 
some of what Ms. Evans has just said. 
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Mr. Powner. A couple thoughts here. One, first of all, I think 
Ms. Evans and OMB, they deserve a lot of credit for all the infor- 
mation we have on the Management Watch List, which are poorly 
planned projects. OK, you want to fix the planning up front. 

But the High-Risk List is not that important. What is important 
is Figure 4 on page 19 in my testimony, which shows those projects 
that have shortfalls and the reasons for the shortfalls. So the No. 
1 reason why we have shortfalls, and this is about performance 
now, it is not just about planning, but we are expending funds and 
we have shortfalls. We are not meeting cost and schedule goals. 
There are 70 projects. What I am hearing is I think the Adminis- 
tration — I am going to cut right to it — they are reluctant to high- 
light projects with shortfalls. 

Senator Carper. Why do you suppose that is? 

Mr. Powner. I think the discussion was along the lines of embar- 
rassing agencies and that type of thing. You can’t fix problems 

Senator Carper. There is a reluctance to embarrass them? 

Mr. Powner. I believe so, but I don’t want — I will let OMB com- 
ment further on that. I don’t think you can fix shortfalls without 
fully disclosing all your problems, so you need to take those short- 
falls — we have 10 percent — we have 70 projects that aren’t meeting 
the 10 percent threshold. That number is greatly understated, and 
I will tell you why. Earned value implementation is weak at many 
agencies, so some of that data is not based on reliable information 
coming from the agencies, and then the point about rebaselining, 
OK. 

If we had transparency on rebaselining — that number right there 
says that 85 percent of the high-risk projects are meeting the 10 
percent cost and schedule threshold. It is likely — that 85 percent, 
you are walking on water if you are a private, public organization, 
85 percent of your projects are meeting within 10 percent. You are 
doing very well, extremely well. 

So we need accurate data there, but we need to disclose all that 
information in order to fix those shortfalls, so that is why we, in 
our testimony, highlighted the important bit of information here 
are the high-risk projects with shortfalls so that we can fix per- 
formance. I actually think this is very consistent with the Adminis- 
tration’s President’s Management Agenda and their E-GOV scor- 
ing. One of their criteria to get to green is you operate within 10 
percent of costs and schedule using earned value techniques. So we 
are all talking the same thing here and we have the same goal. It 
is a matter of disclosing all the weaknesses and then fixing it. 

Senator Carper. Ms. Evans. 

Ms. Evans. I would like to respond to this. 

Senator Carper. Please. 

Ms. Evans. First and foremost, as the Administrator of this, I 
think it is my responsibility to make sure that I have done every- 
thing that I can for the agencies to make sure that the guidance, 
the policy, everything is clear, and what GAO has highlighted is 
that there is a systemic problem throughout the government. So we 
could be focused on the one-offs and let us zero in on this one par- 
ticular project or let us zero in on this. But the tenets of the Presi- 
dent’s Management Agenda is getting good management founda- 
tion processes, with these types of things in place, so that no mat- 
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ter what the project is, no matter what the investment is, the agen- 
cy is going to succeed. 

So when you start looking at the GAO report and you look at 
Table 5, which talks about who is consistent with the best prac- 
tices, and going across and looking at that, what you want — I am 
looking at, what is the problem across the board, and right now, 
what I have gotten from all of this is that I need to do a better 
job of clarifying what is a baseline. How do you do integrated base- 
lines? How do you set certain things up? What are the best prac- 
tices, so that the agencies have the tools to succeed. 

If I am going to sit there and highlight one or two projects and 
say, Agency A is a real screw-up and they are not doing PDQ, we 
are going to drive behavior down underground and these issues of 
rebaselining are going to continue on and on and on because they 
are going to want to hit the mark of being at the 90 percent be- 
cause grades are coming out. So I think it is my responsibility to 
do the things that I can in the first place to make sure that it is 
clear to the agencies. 

Now, once I believe that I have done everything that I can, that 
we have put everything in place, that it is totally clear in the agen- 
cies, the bulk of the agencies are producing, then I have no prob- 
lems providing all the information, putting the transparency out 
there, having accountability, because that now becomes a different 
issue and that is a leadership issue within the agency itself. 

But right now, Mr. Powner has made it very clear that it is an 
immature process, at best, what we have in the agencies, and so 
they should be rewarded for moving forward and being forthcoming 
in the information that they have so that we can address and fix 
the problems, not that I feel passionately about this. 

Senator Carper. Well, I would be disappointed if you didn’t. 

Mr. Powner, do you want to say anything else? Go ahead. 

Mr. Powner. Just to piggyback off of what Ms. Evans was saying 
there, I think the point, too, about the accountability at the agency 
level, agency CIOs are accountable for these fixes. I mean, we need 
to highlight them, and it is not just all in OMB’s lap, but the agen- 
cy CIOs are accountable here. 

Senator Carper. Has OMB issued guidance on rebaselining? 

Ms. Evans. The policy memos that we have in place are high- 
level. They refer back to the Capital Planning Guide. They also 
refer back to A— 11. But I, in preparation for this hearing, have 
gone back all through those, and maybe it is because I live and eat 
and breath this, it seems clear to me, but when I am looking at 
all the results across the board, I think there are other things that 
we can do like putting together a framework document that would 
show them how to do certain things the same way that the CIO 
Council put together a framework for project managers and made 
it very clear what a tier one project was, what a tier two project 
was, and those types of things. I think we need to go down now 
to another level and give them another set of tools in order to be 
able to improve their performance. 

Senator Carper. Mr. Powner, should OMB issue guidance on re- 
baselining? 

Mr. Powner. Yes. There clearly is some clarification that needs 
to occur so that agencies have clear direction going forward. Our 
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report highlighted very simplistic items that the policy should 
cover. For instance, when you establish a new baseline, how do you 
validate it? That is very consistent, Ms. Evans, with the integrated 
baseline reviews that you require and those types of things. So it 
is not inconsistent with things that OMB is already endorsing. 

But the other key part of their policy is it has to be approved by 
management. We found that a lot of policies didn’t even require 
when agencies rebaseline an approval from key managers. That is 
all about the transparency thing so that, in fact, we are getting ac- 
curate data on costs and schedule performance. 

Senator Carper. Mr. Denett, does your office collect information 
on capital investments? 

Mr. Denett. That is through the A— 11 process, through the 
budget. So we get these 300s and the budget side, the Resource 
Management Office culls through those and looks at them to see 
which ones are using earned value management and which ones 
are behind or doing well on cost and schedule. 

Senator Carper. OK. What do you do with that information? 

Mr. Denett. The Resource Management Office looks through 
them and goes back, challenges them, sometimes sends them back, 
asks for more data, asks them what they are doing to fix it, and 
it all rolls up into the budget process. But, important also is the 
caliber of people we have working with this, and as I mentioned 
earlier, we are real pleased that we now have certification require- 
ments for the managers and people assigned to this, contracting of- 
ficers, program managers for the first time have to meet certain ex- 
perience levels and they have to take mandatory training. All of 
that will also assist in doing a better job in this area. 

Senator Carper. All right. Let me just ask you what Congress’ 
responsibilities are, Ms. Evans and Mr. Denett. I think we under- 
stand that GAO is our watchdog to help us better ensure that you 
are doing your jobs well. What do you see as the appropriate role 
for Congress in our oversight responsibility as the Legislative 
Branch? We don’t pretend to be experts on these hundreds of dif- 
ferent IT projects or capital investments. At the same time, we 
have an obligation to authorize and appropriate monies to fund 
these projects. If we don’t have information that is timely and that 
is understandable, we are not able to do a very good job as legisla- 
tors in our oversight capacity. What do you see as our appropriate 
role that would enable us to be constructive? 

I think part of it is to embarrass. I am a person who believes in 
rewarding behavior. I am a big positive reinforcer. But I think the 
time comes when people need a swift kick in the pants and there 
are times when folks just need to be embarrassed. Sometimes, I 
think that is an appropriate role for us. Sometimes, it is an appro- 
priate role for OMB, and certainly for GAO. 

But let me just ask your thoughts about how the Congress, in- 
cluding this Subcommittee, can be more constructive. Mr. Powner, 
any thoughts? 

Mr. Powner. Well, I would agree that you need to shine a spot- 
light on the problems. That is very helpful at times and perhaps 
it is public embarrassment, but it gets action. Hopefully, over time 
with the oversight, we can move down the road here and focus 
more on the solutions. I mean, we know all these projects, we know 
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the agencies, the projects, and the problems with specific projects. 
We know what the issues are. Now we need to put the fixes in 
place to look at how we implement things going forward. I think 
your second panel is a good start, where we start looking at that. 
But again, we are going to probably continue to need to step back 
to make sure that you are getting accurate information because I 
contend that there is still some information here that is under- 
stated. 

Senator Carper. We have talked about some of it, but just be 
more specific. You say you contend that some of the information is 
understated. Just elaborate on that some more. I just want to stay 
on this point. 

Mr. Powner. Out of 800 major IT projects, reporting 70 that 
have performance shortfalls on cost and schedule is an understate- 
ment. 

Senator Carper. All right. Fair enough. Ms. Evans, my first 
question. 

Ms. Evans. If I could point to 

Senator Carper. No, I want you to come back to my question 
what 

Ms. Evans. Well, I am. I am going to give you an example. 

Senator Carper. We spent plenty of time telling you how you 
ought to do your job. 

You have a chance to give us some advice on how we can better 
do our job so you can do your job better. 

Don’t pass up this opportunity. 

Ms. Evans. I know, and so I do believe that there would be indi- 
cators off of the information that we provide to you that by having 
hearings, and I don’t disagree that having public hearings ensures 
transparency and accountability, and you can use this set of hear- 
ings as an example because there is a particular area when we first 
came up here you had no information. You just had gross numbers 
of what we were saying and you kind of had to trust us that we 
were telling you the right thing. 

So I think that if the agencies come up here with the mindset 
along the lines that you are there to achieve the same goal as our- 
selves, that then it becomes a partnership. And I think that the 
hearings themselves and the way that you have approached the 
hearings has allowed for that partnership to happen, and so that 
is why we are at the point with all the information that is out 
there. 

But to your point about — and on one of the charts it has the Cen- 
sus project, which we can all talk about. By having those hearings, 
it does have the agency very focused on it. The way that the poli- 
cies are set up, there are indicators on there that you could see it 
before they have to do a budget amendment. There are things that 
you can do based on pulling this information out that if you had 
a particular area that you were interested in, you can request the 
more detailed information. GAO can request the more detailed in- 
formation from the agency themselves. 

We have set up everything that way so that you can get those 
quarterly reports, the monthly reports, everything that goes behind 
the high-level numbers, and by talking with the agencies or having 
your staff work directly with the agencies without a hearing — I 
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mean, a lot of the work that we have done under the GAO High- 
Risk List in conjunction with GAO and staff up here, those aren’t 
hearings, but the whole idea of having meetings with the staff and 
having to present their goals and their plans and then us reporting 
out on it on a quarterly basis and a 6-month basis really keeps 
agencies on focus, and that is not even a hearing. That is just say- 
ing what you are going to do in the next 6 months and then come 
back up here and present what you have done. 

Senator Carper. All right. Let us talk about solutions. We have 
talked a bit about information technology projects that are making 
it or not making it, how we can get better information, more timely 
information on a regular basis, and information we can under- 
stand. In terms of solutions and solving these problems and moving 
forward, a long time ago, when I was a Naval flight officer, when 
I was on active duty on the West Coast, our job was to hunt for 
Red October. We flew airplanes, 13-man aircraft that tracked So- 
viet nuclear submarines in all the oceans of the world and we had 
a number of squads. We flew an airplane called the P-3. 

We would all go through a training command to be prepared to 
be assigned to our respective squadrons and then we would get to 
our squadrons and we were assembled into 13-man crews and fly 
out our missions. But our respective squadrons didn’t always do an 
especially good job and we had some squadrons that were better 
than others and we had some crews that were better than others. 

And the admiral that was in charge of our operations came up 
with an idea. I think he called it a Tactical Training Team, and es- 
tablished it in the basic training squadrons in which we all initially 
were trained. These Tactical Training Teams would be deployed to 
squadrons that weren’t doing a particularly good job throughout 
the Pacific and Indian Oceans. They would work with us for a 
while and then they would go back home or deployed to assist 
other squadrons on an as-needed basis. There was always plenty 
of work for them to do because we weren’t always uniform — just 
like these IT projects. You have pockets of excellence. We had pock- 
ets of excellence in the work that we did. 

My staff in working with others has come up with an idea. It re- 
minds me of the Tactical Training Team idea. Our folks here call 
it an IT Strike Force. I would like to think that it is a constructive 
and realistic solution, or part of the solution to what we are trying 
to do here. I am not sure, but I hope that it is. 

But Ms. Evans, I believe you may have seen a pre-release draft 
of the bill that I am going to be introducing later today with Sen- 
ators Lieberman and Collins that will allow your office to create, 
if you will, a Tactical Training Team, but really an IT Strike Force. 
It would be comprised of experts from both within government and 
maybe from outside of government, folks who can help agencies 
control some of our IT problems before they become even more un- 
wieldy. 

And the hope here is that we won’t see a repeat performance like 
we have experienced with our Census hand-held devices. This 
Strike Team notion would help agencies control small problems be- 
fore they become billion-dollar problems, as they are today. 

First of all, is this a realistic proposal in your idea? Is this an 
idea that is half-baked? Is this an idea that has promise but needs 
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to be further thought out? What do you like about it? What are 
your concerns about it? I would just start with asking you for your 
comments and then I will ask Mr. Powner and Mr. Denett, as well. 

Ms. Evans. Sir, the initial reaction to it is that it is a solutions- 
oriented approach. It is trying to get to the goal and identifying the 
problems quick enough in a way that we can prevent a situation 
like the Census from happening again. So the initial concept, I be- 
lieve, is a good concept and is a solutions-oriented concept. 

We do something on a very similar basis, but it is not as formal 
as that process would be, now as we identify the problems, but they 
are not as quick. We are not getting ahead of the problem. 

One of the efforts that we have underway now, which I think 
this particular feature would complement, is what we are calling 
the Policy Utilization Assessment effort that we started this year. 
So Mr. Powner has talked a lot about the quality of the data, and 
it is only as good as the agency reports to us. So what we have 
started is an initiative with GSA that actually looks at our policy 
in totality and then does an assessment. 

So, for example, in the cyber security area, we have a policy, 
total policy going from acquisition all the way through how you ac- 
cept it. If I looked at the numbers reported by the agencies, I would 
be at 50 percent. But what we have worked on and what we have 
developed is a methodology that gives us statistical certainty. I am 
really only at 30 percent. 

So based on that, using that and then in complement with what 
is envisioned, I believe, in this bill, I could use this to go into an 
agency and say, OK, you said that you are green. You have all 
these things in place. You haven’t rebaselined, so we are going to 
do an assessment of the policy, not an audit like GAO, but an as- 
sessment, and it would tell me whether they were at 100 percent, 
90 percent, or 60 percent. If they were below a certain threshold, 
then this team could go in, analyze what the issues are, and then 
help the agency to move forward to really realize that 100 percent 
implementation. 

Senator Carper. Mr. Powner, this notion, this IT Strike Team 
notion, is it half-baked? Is it more than half-baked? One of the 
things I think you mentioned in your testimony, we have these IT 
projects that we know are problematic. They appear on a High-Risk 
List or a Management Watch List, not just once, but again and 
again and again, for extended periods of time, for months and in 
some cases for years. And maybe that might be an appropriate 
place or instance in which to deploy these teams. 

Mr. Powner. Yes. I think the idea is a good one from this point. 
Agencies need help, and if you look internally to some agencies, 
they set up similar — they refer to them as Centers of Excellence 
where they can go to for help in establishing contracts. How do we 
establish a baseline estimate up front? How do we define our re- 
quirements? How do we manage risk? And so you put the appro- 
priate processes in place. And if you look at those things, a lot of 
those are the root causes why we have projects on this list, why 
we rebaseline. We got the requirements creep. Requirements creep 
is out of control so we have rebaselined. That was the No. 1 reason 
we found. 

Senator Carper. Is that right? OK. I am not surprised. 
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Mr. Powner. No, exactly. So if you put this group together, I can 
tell you right out of the gate what some of the areas you want to 
focus on: Getting good estimates up front, defining your require- 
ments well, putting in place a sound risk management program, 
and overseeing contractors, and there are probably some other 
thoughts, but those are four areas where we have major weak- 
nesses across the board. 

So I think it is a good idea because if you focus on solutions and 
areas where they can go for help — but what is important is that 
group from a central government point of view, there will be a lot 
of pressure on that group because there is a lot of help that is 
needed. What is important is to establish that group and then to 
have that somewhat replicated in agencies, too. And we already 
have some of that. I think DOD, FAA, IRS, some of the big organi- 
zations, you will see pockets of that occurring. 

So it is also important to do that centrally, but to make sure that 
it gets replicated and not to lose sight, too, that it is about the peo- 
ple, also, not just the processes but the people. We need to bolster 
our workforce. I think Ms. Evans comes out with an annual report 
showing that we need help in the PM area, with architects, engi- 
neers, those types of folks. 

One other item that I would like to highlight is if you look at IRS 
as an example, where they had some successes, they used what is 
called Critical Position Pay Authority, where you actually can pay 
above some of the Federal limits to attract some better folks. So 
in addition to these processes and strike forces, we need to con- 
tinue to bolster our workforce through some of those existing proc- 
esses. 

Senator Carper. All right. Well, I think I have probably kept you 
here long enough. I wish that more of my colleagues were here 
with us today. I know Dr. Coburn has a lot on his plate these days, 
but is keenly interested in these issues. He has not gone away. He 
is going to be around for a while. He has self-imposed a 12-year 
time limit if the voters of his State concur with that, and they 
might. 

I am going to be around for at least another 4 years and my 
guess is that we will continue to work together on this Sub- 
committee for a while, so our interest in this issue, in these issues, 
as well as a broader range of issues that involve concerns about 
just how wisely we are spending our resources, we are going to stay 
on these, but we are going to stay on this one, as well. 

I think we are making some progress, but God knows we need 
to make more. There is plenty of work for us to do here. 

Mr. Denett. 

Mr. Denett. I agree with what my colleagues have said. I would 
like to tack on, we do have an initiative called the SHINE initia- 
tive, because so often 

Senator Carper. The SHINE? Is that an acronym or what is 
that? 

Mr. Denett. It is just meaning shine, give people a chance to 
shine and look good. Too often, they are focused on, when they get 
behind schedule and over cost, and that is appropriate, but we 
don’t often enough praise those that are within budget and on 
schedule. So SHINE is an attempt to give recognition to programs 
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and employees that are doing well. So we gather them in, give 
them some praise 

Senator Carper. That is good. 

Mr. Denett. I think it would be helpful and would even like to 
approach you when we have our next round of those to see if you 
would be willing to participate in honoring some of those special 
projects. 

Senator Carper. I would be pleased to do that. I would be 
pleased to do that. What we used to say in the Navy, praise in pub- 
lic, reprimand in private. We would be pleased to join in the praise 
for some of these folks. But every now and then we like to, around 
here, we like to reprimand in public, too. It has a salutary effect. 
[Laughter.] 

All right. I suspect you are going to have some follow-up ques- 
tions for the record. I would just ask that you respond to them 
promptly, as you always do. 

Thank you for your stewardship here, and Mr. Powner, a real 
special thanks to you and your folks at GAO who are a big help 
to us in this area. Thank you. 

All right. Mr. Grasso, Dr. Brown, Dr. Jarrett, we are happy to 
see you, Tom Jarrett. Welcome. 

Our second panel today begins with A1 Grasso. Mr. Grasso is 
President and Chief Executive Officer of the MITRE Corporation, 
a not-for-profit organization chartered to work in the public inter- 
est. I think you all have been involved, as I understand it, in doing 
some work on the Census project and we are very grateful for the 
work that you are doing there. At MITRE, Mr. Grasso is respon- 
sible for developing and leading the corporation’s overall strategic 
and business operations. MITRE has been involved in helping to 
oversee multiple complex and high-risk IT investments in the Fed- 
eral Government. You have had plenty to do. 

Our next witness is Dr. Norm Brown, and I couldn’t help but no- 
tice Dr. Brown sitting in the first row behind the witnesses on our 
earlier panel. Dr. Brown did a lot of head nodding one way or the 
other, so now we will let you actually lend a voice to all the body 
language that you were sharing with our staff and me earlier. 

But he is Executive Director for the Center for Program Trans- 
formation. For over 30 years, I am told, Dr. Brown has served as 
a commercial program manager and program management trouble- 
shooter, and you have also served as the Assistant Secretary of — 
did you serve as the Assistant Secretary or in the office of the As- 
sistant Secretary of the Navy — in the Navy, good for you — where 
you worked across all military branches to bring troubled projects 
back on track. In addition, I am told that you represented the mili- 
tary services on the 2000 Defense Science Board with the mission 
of solving why risky IT investments spiral out of control. 

And our last witness, I am tempted to say saving the best for 
last — he is a home boy — but Tom Jarrett, whom I have been privi- 
leged to know for a long time, Secretary of our Department of Tech- 
nology and Information for the State of Delaware. Mr. Jarrett is no 
stranger in testifying before this Subcommittee. Previously, he tes- 
tified as President — President, that is a job a lot of my colleagues 
would like to have — President of the National Association of State 
Chief Information Officers on issues relating to cyber security when 
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Senator Coburn was Chairman of this Subcommittee. Mr. Jarrett 
oversees an IT investment budget of over $200 million and has 
achieved 90 percent of cost, schedule, and performance goals for the 
past 7 years as the Chief Information Officer of Delaware. 

When I was governor, we made progress on a whole lot of dif- 
ferent fronts overall in our education and welfare system. I was 
very pleased with much that we accomplished, such as job creation 
and job preservation. One of the areas that we made some 
progress, but maybe not enough, is the area of our government that 
Mr. Jarrett now leads. He has taken our State clearly to the next 
level. I think one of the very proud things that our current gov- 
ernor can take credit for is the work that Mr. Jarrett and his folks 
have done in his department. So, welcome. In terms of best prac- 
tices, we think you are one. 

Mr. Jarrett. Thank you. 

Senator Carper. Mr. Grasso, I am going to recognize you for 
your statement, and again, I would ask you to use about 5 minutes. 
If you go a little over, that is ah right. And then once all of our 
witnesses have testified, we will come back and ask some ques- 
tions. Your entire statement will be made part of the record, so 
please proceed. 

TESTIMONY OF ALFRED GRASSO , 1 PRESIDENT AND CHIEF 
EXECUTIVE OFFICER, MITRE CORPORATION 

Mr. Grasso. Thank you, Chairman Carper. Thank you for afford- 
ing me the opportunity to appear before this Subcommittee. I fear 
that many of the remarks I had prepared have already been stated 
this morning, but I hope to reemphasize some of those key points. 

Senator Carper. Around here, we talk about an echo effect. It is 
really hard to cut through the media and to actually deliver a mes- 
sage to constituents, to voters. If you are the President, you have 
a big megaphone and you have a lot of echo effect from your cabi- 
net secretaries and others who work for the Administration, so you 
have a good echo effect. But things have to be said over and over 
and over again in order for them to get through, including to people 
like us up here. So a little repetition is not bad. 

Mr. Grasso. Thank you. Our company’s 50 years of experience, 
contributions, and accomplishments has given us a perspective that 
I believe is highly relevant to the topic of information technology 
planning and management. From the early days of the SAGE air 
defense system to present-day deployment of advanced command 
and control and business modernization systems, MITRE has been 
witness to great successes and, similarly, to great disappointments. 
We are honored to be asked to share our lessons and insights with 
your Subcommittee. 

Federal IT programs operate in an environment of rapid tech- 
nology evolution, where some system components become obsolete 
before the program completes. This pace of technology change chal- 
lenges program teams to keep their technical skill base current. IT 
systems and business processes are increasingly interconnected 
within and across agencies, making it hard to achieve consensus on 
vision, operational concepts, and requirements. The Federal Gov- 
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ernment’s stretched fiscal and human resources further complicate 
the situation. 

The net effect is the widespread failure of many programs to de- 
liver on time and on budget, with only a few notable exceptions 
where programs are able to overcome these challenges and succeed. 
Our experience leads me to comment on several critical areas and 
to offer three steps for improvement. 

My first comment pertains to governance. Governance relates to 
decisions that define expectations, grant power, assign account- 
ability, or verify performance. Effective governance comprises con- 
sistent management cohesive policies, processes, and decision 
rights for a given area of responsibility. Governance becomes in- 
creasingly complicated as programs and processes cross organiza- 
tional boundaries and intersect multiple governing bodies. Authori- 
ties and responsibilities become ambiguous and program managers 
are disenfranchised. It is often said that the debate begins in gov- 
ernment once the decision is made. 

Successful programs must have unambiguous governance. Deci- 
sion making authority 

Senator Carper. Say that again. Successful programs must have 
what? 

Mr. Grasso. Unambiguous governance. 

Senator Carper. OK. 

Mr. Grasso. Decision making authority and accountability that 
address the implications of intersecting organizations must be 
clearly defined at the onset. These authorities must encompass the 
areas of budget and finance, investment portfolio management, 
business process, and program and project management. 

My second comment pertains to requirements, an equally impor- 
tant consideration. Requirements, reality, and flux are often recog- 
nized as the root cause of program rebaselining. Rebaselining is not 
necessarily a dirty word, but a necessary part of delivering capa- 
bilities that meet users’ needs. Requirements are too often deter- 
mined in the absence of cost, schedule, and technology risk consid- 
eration, and once determined, they are very difficult to change. 

The biggest difference between successful commercial IT develop- 
ments and troubled government IT acquisitions is how require- 
ments are managed. Successful commercial IT developers handle 
requirements with great caution. If a certain requirement ad- 
versely drives cost, performance, or schedule, it is quickly modified 
or eliminated. This does not happen in a typical government IT ac- 
quisition. Time to market is a competitive driver in the commercial 
marketplace, and I would submit it is as important, if not more so, 
in a world where adversary capabilities change as quickly as the 
technology cycle. System requirements must be considered living, 
but managed with a controlled process to use regular trade-off 
analyses to determine the value of change. 

My final comment addressees program management practices. 
Successful programs are characterized by a strong government Pro- 
gram Management Office (PMO), capable of a peer relationship 
with the contractor on systems engineering and program manage- 
ment issues. With a strong and capable PMO, the government has 
the capability to make informed decisions and manage the risk in 
acquisition programs. 
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A key function of a strong PMO is best described by the meta- 
phor of an architect’s relationship with the user and the builder of 
a building. The architect is the user’s agent as independent of the 
builder. The architect works to understand the user’s operational 
needs and translate them into technical requirements enabling 
builders to develop the needed capability. The architect evaluates 
development feasibility and performs an independent conceptual 
design and cost estimate. These architect functions enable the user 
to make informed cost and capability tradeoffs and prioritize re- 
quirements. The architect is accountable to the user to ensure that 
delivered capability meets the user’s highest priority needs within 
the constraints imposed by available technology, funding, and time. 

I offer the following recommendations based on our experience 
with these issues. First, change the tone and tenor of oversight to 
focus equally on programs that have gone from bad to good and 
good to great, to reveal best practices which then can be applied 
more broadly. No program is without risk. We should all be more 
interested in those programs that have managed the risks well and 
harvest those results for the betterment of the larger set of pro- 
grams. 

Second, to navigate the dynamics of uncertainty of today’s envi- 
ronment, IT programs are best structured as a portfolio with inter- 
nal planning and management flexibility. Oversight should focus 
on the long-term funding envelope and the overall capabilities to be 
delivered. This allows flexibility at the program level to make in- 
formed trade-off decisions and to concentrate on manageably-sized 
increments that deliver capabilities in shorter time frames. This 
approach makes it easier for programs to demonstrate success or 
to fail early, which is valuable if a program is put in place and 
funded contingencies. It also puts capabilities in the hands of the 
users more quickly. This incremental approach is the norm in com- 
mercial practice. 

Third, Congress should continue to support and refine programs 
such as the DOD’s Highly Qualified Experts program and, as Dave 
Powner mentioned earlier, the IRS’s Critical Pay Authority that 
helps attract and retain critical government professionals. Addi- 
tionally, the IRS’s pay-for-performance program has helped moti- 
vate performance aligned to outcomes. These are valuable tools 
that address the capacity, capabilities, and incentives needed to 
manage effective programs. We encourage the Congress to look to 
these as models, streamline their execution, and broaden their ap- 
plication government-wide. 

I request that my prepared statement be included in the record 
and I would be pleased to answer questions. 

Senator Carper. Your entire statement will be included in the 
record. That was an excellent statement, an excellent summary. 
Thank you very much. 

Mr. Grasso. Thank you. 

Senator Carper. Dr. Brown, welcome. 
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TESTIMONY OF NORM V. BROWN , 1 EXECUTIVE DIRECTOR, 
CENTER FOR PROGRAM TRANSFORMATION 

Mr. Brown. Good morning, Chairman Carper. First, let me con- 
gratulate you for holding this hearing since literally billions of tax- 
payer dollars are wasted every year in poorly managed IT. Clearly, 
you are onto something important. 

Vice Admiral Jerry 0. Tuttle, Retired, the former Deputy Chief 
of Naval Operations for C4I and an icon for naval computing and 
net-centric warfare would counsel, “Lead, follow, or get out of the 
way.” Thank you, Senators. Thank you for leading the way. As we 
say, bravo zulu. 

Let us be clear about one thing. Although it is difficult to effec- 
tively manage a large-scale project, on the other hand, producing 
a large-scale IT train wreck is easy. The good news is that wrecks 
can be avoided by effectively using best practices. 

Today, I would like to briefly offer actions that government de- 
partments and agencies, OMB, and Congress can take to prevent 
wrecks. At its core, these actions address rapidly achievable im- 
provements. 

I will begin with a structural observation. Much is expected of 
each agency CIO. Many have responsibility without real authority. 
Many Federal departments include numerous essentially inde- 
pendent fiefdoms because Congress has so arranged it, fiefdoms 
independently funded by Congress. The Pentagon rule is, he that 
has got the gold makes the rules. So, too, in Federal agencies. Al- 
though I don’t today have any solutions to offer, I would be happy 
to work with your staff. 

Next, an observation regarding those IT problems in project man- 
agement and oversight. When OMB testified before you last Sep- 
tember, they expressed a recent interest in IT program execution, 
and that is a very good thing, but IT programs simply don’t man- 
age themselves. 

From my understanding and as we have heard this morning, 
very few agencies have much in the way of any real IT program 
management and oversight. Earned value is held up as a do-all sil- 
ver bullet solution, yet little is done to prevent the easy gaming 
and corruption that earned value is vulnerable to, and associated 
rebaselining may lack the transparency needed to ensure effective 
oversight. Far too much is expected of earned value. Although 
earned value is a powerful visibility technique that supports pro- 
gram management, earned value cannot replace program manage- 
ment. 

Unfortunately, there seems little in the land of government IT 
program management that implements the needed essential tech- 
niques of managing risk, requirements, and change, or integrated 
baseline reviews. As a solution, I would recommend that each agen- 
cy be required to actually have real program management and 
oversight, that they focus on implementing the important critical 
details with minimum overhead, and that they identify remaining 
weaknesses and vulnerabilities. 

Transparency is dandy, but it must be converted to visibility to 
be useful. A transparent contractor can deliver a 53-foot truck full 
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of boxes of data, but what you really need is only the bottom-line 
information. What is needed is true visibility of IT project health 
and progress in near real time. 

Since earned value seems to be the only principal visibility tech- 
nique relied on by OMB and the agencies, I would propose a more 
comprehensive visibility product, let us say an Exhibit 350, to pro- 
vide real project visibility indicators monthly, primarily for the pro- 
gram manager, with quarterly simplified versions for agency and 
component CIOs, OMB, and Congress. I will be happy to work with 
your staff and OMB on this. 

Tracking schedule progress is not easy. One reason for schedule 
surprises is that it is pressure to meet schedule increases. The 
hard-to-do things are kicked down the road, with difficult, 
uncompleted requirements now moved into the future, a future 
which was not planned to receive it. As it turns out, this unplanned 
future work will now require a successive series of miracles to be 
accomplished in order to complete the development on time and on 
budget. Don’t bet on the miracles happening. 

To motivate agencies to focus on IT project management and 
oversight, I would recommend adapting the Nunn-McCurdy notifi- 
cation process, not that it is a great visibility technique; it is not. 
But it, in fact, serves as a powerful motivator, as something really 
to be avoided. As little else can do, it gives a clear focus to the busi- 
ness of cost and schedule containment. 

We have a serious problem regarding people. It is difficult to re- 
ward good talent, to hire good talent, and to train good talent. We 
expect CIOs and our IT personnel to do more with less and then 
give them less as if to prove the point. OMB needs to address this 
people issue as a priority. 

Training project personnel to effectively implement fundamental 
processes is minimal. While DOD has a certificate program in IT 
program management, GSA long ago disbanded their excellent 
similar IT Trail Boss program for civilian agencies. The various 
communities charged with making IT development work have re- 
ceived essentially no education, training, or any certification to do 
what they are asked to do. No training, no education, no certifi- 
cation, and no experience — it is a wonder we do as well as we do, 
as bad as it is. 

Apropos of your remarks earlier this morning about pulling the 
plug, a program termination process should be considered. The 
former Assistant Secretary of the Army for Acquisition, the Hon. 
Claude Bolton, is a strong proponent of terminating projects if 
measurable outcomes cannot be achieved within the agreed-to pro- 
gram cost schedule and performance baselines. The DOD has no 
formal decisionmaking process or policies to terminate programs, 
and I am not aware of any for other agencies. I think it wise and 
prudent to consider including such a process among ways to im- 
prove the government’s IT acquisition process. 

Contract incentives and other considerations are important, and 
I discuss them further in my formal testimony. 

That concludes my summary and I will be happy to take ques- 
tions at the appropriate time. 
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Senator Carper. Again, another excellent testimony. Thank you 
for the thought that you put into it. Thank you for your years of 
service, too, and your counsel here today. 

Mr. Jarrett, you are recognized. Please proceed. Welcome. 

TESTIMONY OF THOMAS M. JARRETT , 1 SECRETARY, DEPART- 
MENT OF TECHNOLOGY AND INFORMATION, STATE OF 

DELAWARE 

Mr. Jarrett. Chairman Carper, Ranking Member Coburn, and 
Members of the Subcommittee, thank you for the opportunity to be 
here today. As Secretary of Delaware’s Department of Technology 
and Information, I can well appreciate the complexity, the chal- 
lenges, and the significant responsibilities associated with man- 
aging information technology projects in an investments portfolio 
that cuts across many agencies, and in Delaware’s case, all three 
branches of government. 

Albeit in a much smaller scale than the Federal Government, 
Delaware and other State Governments are faced with similar con- 
cerns regarding IT project management. In fact, Delaware’s De- 
partment of Technology and Information was established in part 
because of ongoing IT project delays and cost overruns that Dela- 
ware was experiencing. Delaware’s centralized IT structure charges 
our agency with direct oversight and approval on nearly $200 mil- 
lion in active IT projects. 

As a new agency, we were able to develop and employ new ap- 
proaches to IT project management that incorporate many best 
practices from private industry and others in the government sec- 
tor. While our methods are under constant review for improvement, 
we are enjoying some significant progresses. We have an excellent 
track record of delivering much-needed IT solutions that are on 
time and on budget. 

There are no silver bullets, no one or two changes that you can 
point to for perfect project management. Instead, there are many 
small improvements that we have made that, in the aggregate, are 
making the difference for Delaware. I would like to take a few min- 
utes to give you a high-level view of a project life cycle using the 
Delaware model. In addition, I would like to point out what I be- 
lieve are the significant processes and procedures that we have put 
in place to successfully manage our portfolio. 

Like the Federal model, we require agencies to submit a business 
case that addresses the major items we believe help ensure a 
project’s success. Our model includes the following major areas: 
Risk management, processes reengineering, architectural review, 
resource and funding availability, project management oversight, 
organizational change management, needs assessment, custom- 
ization requirements, disaster recovery levels, and management 
and executive sponsorship. 

Recommended projects are forwarded for the concurrence of the 
State CIO before they move ahead in our process. This rec- 
ommendation usually includes ongoing funding contingent on meet- 
ing project milestones. Once approved, DTI works with the agency 
customer to develop full and complete requirements so a request 
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for proposals can be released to secure vendor bids in order to meet 
the needs of the project. DTI stays involved to help the agency 
make the vendor selection and to structure a contract that ensures 
the project is delivered on time and on budget. 

I cannot emphasize enough how critical the requirements gath- 
ering process is to the project’s ultimate success or failure. The 
Delaware model does not allow for requirements gathering to be 
conducted solely by the vendor. There must be an active involve- 
ment by the DTI project management team. There is an old saying, 
“The customer doesn’t know what they want until you give them 
what they ask for.” Requirement gathering is critical to the process 
and helps alleviate scope, time, and budget creep if it is done cor- 
rectly. 

At the start of a new project, a nationally certified program or 
project manager is assigned, as well as a certified organizational 
change management team, to run parallel courses in managing our 
projects. It is not enough to be certified. We take extra care in se- 
lecting the people to fill these positions, as they are essential to the 
success of the project. We count on these folks to do what we call 
inflicting discipline and structure to these projects. 

It is important to note that DTI manages Delaware’s IT projects, 
not the vendors assigned to them. While we demand the vendor as- 
sign a certified and talented project manager, our State folks man- 
age the project. 

Further, I would like to highlight our change management proc- 
ess. Change management starts with the review of existing busi- 
ness processes and is focused on preparing the organization for the 
cultural changes that are a part of any major project. According to 
a recent study conducted by the National Association of State Chief 
Information Officers, which represents State CIOs across the 50 
States, 80 percent of major IT project failures can be directly at- 
tributed to a lack of change management. Too often, the employees 
who will actually use the new application are left out of the project 
process, and when this takes place, it is almost certain that the 
project is doomed to fail or under-perform. 

Another major element of our success is how we manage the 
overall process. All of Delaware’s major IT projects have executive 
sponsors. Executive sponsors include high-level managers from the 
agencies involved as well as key executives from DTI and our Of- 
fice of Management and Budget. The executive sponsors hold reg- 
ular monitoring and update meetings and provide high-level over- 
sight. When difficult project decisions need to be made, the execu- 
tive sponsors are aware of the issues and have the authority to 
make critical decisions on whether or not to keep the project mov- 
ing along. 

A real example is our present project involving all of Delaware’s 
courts. Due to the lack of employee training and readiness, the ex- 
ecutive sponsors recommended that the project be paused so that 
the training could take place. In fact, Delaware’s Chief Justice him- 
self invoked a 6-month pause in recognition of the need for court 
employees to be prepared if the project was to succeed. 

Besides ongoing project meetings, all projects in the State’s port- 
folio are reviewed weekly by our Project Management Office and bi- 
monthly at a workload management meeting where project man- 
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agers present their projects’ status to the assembled DTI senior 
managers. Opportunities for improvement or needed adjustments 
are vetted in an open forum with the goal of keeping the project 
on track and transparent. 

Our agency is committed to project transparency all the way to 
our legislature and our governor. We believe in continuous informa- 
tion exchange and dialogue with our elected representatives so that 
they are educated and aware of the complexity of major IT projects. 
Providing information up front, even when it may be painful, is far 
better than saving unpleasant surprises for yearly budget sessions. 
Bad news does not age well. 

I wish I could tell you that everything is humming along per- 
fectly and that the processes that we have put in place are the final 
answer, but I can’t, as we have many challenges similar to those 
being talked about today. However, we believe that by tightly man- 
aging these challenges, we can deliver projects on time and on 
budget. Although our project tracking system is complex, in our 
communication with stakeholders, we use a high-level red/green/ 
yellow scorecard to inform them how the projects are developing. 

All projects have or will have ongoing issues that require senior 
management attention. Under the Delaware model, we believe that 
providing the facts, good or bad, to all stakeholders, including the 
legislature, is the proper policy, and most importantly, lives up to 
one of my agency’s core values of integrity. Our vendors are held 
accountable to deliver what they said they would, as well, and we 
call the facts as we see them. 

I thank you for the opportunity to testify today and would be 
happy to answer any questions. 

Senator Carper. Thank you. Thanks not just for your testimony, 
but for the terrific leadership that you provide for our State and 
have shared some of the fruits of that leadership with us today. 

I think I would like to ask, first of all, Mr. Grasso and Dr. Brown 
a question and ask you to reflect on what we have heard from Mr. 
Jarrett today. I have oftentimes described the 50 States as labora- 
tories of democracy and the belief that somewhere in those 50 
States, somebody has come up with a solution that will help us 
solve a number of the problems we face at the Federal level, and 
the same is true of the private sector. Somebody has figured out 
how to solve most of the problems, not all, but most of the prob- 
lems that we face as a Nation and we just have to figure that out 
and be able to grow them to size or to scale. 

What did you hear from Mr. Jarrett in talking about Delaware, 
the way we operate in our State, that might be applicable to us 
here at the Federal level? What are some good lessons learned that 
you think we could take from his testimony and apply them to the 
Federal Government? 

Mr. Grasso. I think a strong element that I heard in the suc- 
cesses that we have seen in Delaware is the investment, again, 
that is made in the project management team, in the strength of 
that project management team. We heard that the project manager 
is certified, but it goes much further than just simple certification. 
There is a continuous development effort that is required. 

With a strong project management team, that project manage- 
ment team can be held accountable to basically continue to own the 
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technical baseline of the program and not just basically contract it 
away such that you just get what you get. The team is informed 
throughout the process, is able to establish that peer relationship 
which I mentioned in my testimony with the contractor, and that 
allows for the successful acquisition. 

Requirements are important. We heard that. The level at which 
requirements are stated are very important. All too often, we detail 
requirements down to a level that we believe perhaps causes some 
of the rebaselining that we see because we specify things in terms 
of technology as opposed to in terms of outcomes. Although he 
didn’t say it, the importance of requirements and the emphasis 
placed on requirements, I suspect they may have gotten that right 
in terms of the level by which they establish requirements. 

So I think a combination of the strength of the program manage- 
ment team, the investment made in those people to keep them 
highly qualified, and the ability to deal with requirements at the 
right level have achieved some of the successes. 

Senator Carper. Thanks for that analysis. 

Dr. Brown, what did you hear from Mr. Jarrett’s testimony that 
you think might apply to us at the Federal level? 

Mr. Brown. First, let me echo what A1 Grasso has said. I think 
the emphasis on requirements development and control and change 
control is absolutely fundamental and determines much of our cost 
and schedule problems that we see. 

And I think there is one other thing that Mr. Jarrett hasn’t fully 
articulated to you that is very relevant, which is his process of pay- 
ing their IT folks. Maybe you could talk more about that. 

Senator Carper. You actually pay them? 

Mr. Jarrett. Yes, imagine, they want to get paid. [Laughter.] 

As you know, Mr. Chairman, a number of years ago, the legisla- 
ture allowed Delaware, and I still believe the only State today that 
has ever done it, to take the entire IT organization and convert it 
from a civil service structure to a non-civil service structure, and 
along with that, they gave me the ability to write our own com- 
pensation structure. So I think we are one of the few, if only, State 
agencies in the country that actually pays its IT people to the mar- 
ket, and what that means is that we have very qualified people, 
and I am happy to say we also have the highest retention rate of 
any agency in the State. 

Senator Carper. I used to be State Treasurer. I remember a time 
early on when we thought that we hired people, usually fairly 
young people out of school, and trained them so that they could be 
hired away by the private sector or some other employer. That 
sounds like that is not the case anymore, is it? 

Mr. Jarrett. That is not the case, not in my department. 

Senator Carper. OK, good. Dr. Brown, do you want to add any- 
thing? 

Mr. Brown. Sure. One thing that you had talked about that par- 
ticularly resonated with me was the concept of the tactical training 
teams, and something that I had started at the Navy Department 
and later became DOD was the Software Program Managers Net- 
work. We had 10,000 members across the country. We supported 
over 200 programs in helping them. We had a stable of these peo- 
ple that we would call tactical trainers, experts in a wide range of 
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subject matter experts, over 100 of these people, and as programs 
need, they could just call in the tactical team. 

If they needed help on a task activity network to support earned 
value or if they needed to better understand how to identify risks 
in the program or had a plan for testing or had a better oversight 
of what the contractor is doing or incentives, anything like that, 
they could call in the team. That is very consistent with your IT 
support team, and I think that is a fundamentally important thing 
to do. 

Senator Carper. All right. Mr. Grasso and Dr. Brown have been 
good enough to comment on your testimony. Let me just ask you 
if you would do the same with respect to some of what they said 
and just reinforce the relevance of their counsel to us. 

Mr. Jarrett. Well, I was telling Dr. Brown before we started, I 
was so pleased when I had a chance to read and hear what they 
both had said because I kind of threw up my arms and said, gee, 
we are doing that and have been doing it for some time and I think 
it proves out in a couple of areas. 

One is the requirements. People get sick about hearing about 
that, but in fact, what we have found and where we have spent all 
of our time is on the front end of the project process, not on the 
back end, which means that if you are going to get them right, 
then you have to determine what the requirements are. What we 
learned in the very beginning is that is not unlike — and you need 
actually qualified and trained people to be able to do that. That is 
not as simple as going out to an agency and saying, “OK, tell me 
what it is that you need,” because what we have found is that in 
a lot of cases, they can’t articulate what they need, at least not in 
a way that when you are looking to replace a system and do that. 
So you have to help them through that process. So we have spent 
a lot of time doing that. 

The other is in the change management area. I think a lot of 
States and a lot of folks are moving into the areas of PMOs, Project 
Management Offices. Something that we have spent an awful lot 
of time on is, again, in the organizational change management, and 
I will give you an example. We are doing a new financial system 
in the State. We are spending a lot of our time not on the technical 
aspects but in dealing with each of the agencies so that we can 
help the users kind of work through the processes and the changes 
that this new project or this new system is going to provide to 
them. Most times, we don’t spend time doing that. 

So I hear both of those areas in the testimony that was given, 
and from my perspective, I think it is well thought out and some- 
thing that I think the Federal Government could do, as well. 

Senator Carper. Mr. Grasso, did you have a point you wanted 
to make? 

Mr. Grasso. I might just offer an additional remark. 

Senator Carper. Before you do that, in your testimony, Mr. 
Jarrett, you said there is an old saying the customer doesn’t know 
what they want until you give them what they have asked for. 
There is a lot of wisdom in that. 

Mr. Grasso. In my written statement, there are a couple of ref- 
erences that I would just point to which emphasize the require- 
ments point, and one of those references points to the 32 programs 
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at NASA that were evaluated and the up-front investment and the 
program overrun result. What you see is when the definition phase 
as a percent of the total program is greater than 10 percent, what 
you find is that the program overrun is typically less than 20 per- 
cent. 

Senator Carper. Say that one more time. I want to make sure 
I got it. 

Mr. Grasso. The graph refers to 32 NASA programs that have 
been evaluated and it takes a look at the definition phase of those 
programs as a percent of the total program. And when the defini- 
tion phase of the program is 10 percent or greater, what you find 
is that the program overrun is 20 percent or less. 

Senator Carper. OK. Good. 

Mr. Grasso. And that suggests that considerate thought is being 
given to the requirements up front and thus less change is required 
later on. 

Senator Carper. All right. Thank you. My staff has been good 
enough to prepare a number of questions, and I am just going to 
take a moment and look at those and I suspect we will use these 
as a point of departure for some further discussion. In some ways, 
the kind of discussion that we are having here is actually more 
helpful than not. 

I have asked you to sort of reflect on your respective testimonies 
here today. Go back with me to the first panel and some of the 
things that you heard from the first panel, some of the things that 
we discussed. What were some of the things that struck you that 
were especially important or maybe something that you wanted to 
add something that wasn’t said, or maybe an answer that wasn’t 
given, or at least wasn’t given as well as it might have been? Does 
anything stand out for you, referring back to the first panel’s testi- 
mony? 

Mr. Grasso. I would offer one remark because I can’t stop look- 
ing at the report card. Having several daughters in high school, I 
have encouraged my daughters to take Advanced Placement class- 
es, whereas I know some of her friends are taking lesser chal- 
lenging classes, if you will, and 

Senator Carper. What grades are your daughters going to be in 
this fall? 

Mr. Grasso. Actually, my oldest is going to the University of Vir- 
ginia this year, so she is just graduating, and my middle daughter 
is a rising 11th grader. 

Senator Carper. OK. We have two boys and our youngest son 
just graduated from the Charter School of Wilmington and he is 
going off to William and Mary and we know all about Advanced 
Placement tests. 

We just got our results about a week ago, high fives all around. 
[Laughter.] 

Mr. Grasso. So you know the amount of work that goes into pre- 
paring — 

Senator Carper. Yes, we do. 

Mr. Grasso [continuing]. And the level of effort that is required. 
The report cards are a great start, but at the same time, all IT pro- 
grams aren’t borne of the same level of complexity, the same level 
of interdependency and challenge. So as we step back and I look 
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at the report cards, it begs the question of what are the correla- 
tions, if you will, to those grades. 

As I look at some of those organizations, some of those organiza- 
tions are more insular than other organizations, if you will, and 
there are fewer interdependencies, and as a result, what you find 
is the complexity of the IT system is not necessarily as challenging 
as an enterprise system that touches a number of different organi- 
zations, crosses boundaries, and has unclear governance. 

So a comment that I will make is a report card as such, I don’t 
know if those are AP classes or if those are substandard classes. 
So an F on AP class may not necessarily give you all of the details 
that you need to respond. 

Senator Carper. Well, you could not have picked a better anal- 
ogy- 

Mr. Brown. My view is that those don’t represent AP classes. 
[Laughter.] 

And with regard to what Ms. Evans and Dave Powner were say- 
ing about earned value management and particularly the problem 
of rebaselining, DOD deals with that in a very direct and forthright 
manner, which is just to track all the costs and the curve and what 
you see is a curve that starts going down and then the rebaseline 
goes up and goes up and it is just this downward spiral, a death 
spiral. And it is very easy to track what is happening when you see 
that picture, and that would be very easy for OMB to provide to 
you. I don’t know if they are willing to do that, but they should 
have that information when they say — I don’t know what agencies 
exactly report to them in their private conversations, but that 
should be certainly public information. 

Senator Carper. OK. 

Mr. Grasso. The 1987 Defense Science Board had an interesting 
conclusion and it said, technology is not our problem, management 
is, and they were talking about software problems, and hope- 
fully — 

Senator Carper. Who said that? 

Mr. Brown. The 1987 Defense Science Board study on software. 

Senator Carper. OK. Mr. Grasso, the folks at MITRE have been 
good to help us in the Bureau of the Census. You tried to recover 
from a very bad situation. You may not be intimately familiar with 
this, but maybe you are, and to the extent that you have some fa- 
miliarity with how the Census Bureau got into this mess as they 
prepared for and approached the 2010 Census, where do you think 
they went wrong and what lessons can we take from that experi- 
ence to make sure that whether it is the 2020 Census or other IT 
projects, how we don’t let this kind of thing happen again? 

As it has turned out, it is an enormously expensive problem and 
we are going to not only end up spending more money, I don’t know 
that we will get a better product. We will probably not get as good 
of a product in the end, but we will have spent a lot more money, 
a very unhappy situation. But we appreciate what you all have 
done to try to minimize the loss and get us going in the right direc- 
tion. But what are some lessons learned? 

Mr. Grasso. I think the two lessons have already been described 
today. The first is requirements. In the case of Census, as you step 
back, there certainly has been a significant number of require- 
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ments changes throughout the process which has led to changes 
with the contractor. In the time period in which this all has oc- 
curred, the technology has evolved, so the solution that was envi- 
sioned some time ago perhaps is not necessarily the best today in 
terms of the architectural basis for that solution. So the program 
was not able to evolve as quickly as the technology was able to 
evolve. 

The second part of that is as all of this is evolving and all of 
these changes are being made, the strength of the program office, 
quite frankly, I would say, could have used some building up in the 
early stages. That has occurred as a result of the response to the 
issues that they have been confronted with most recently, but quite 
frankly, I would say that many of the responsibilities that should 
have been within the program office in establishing the technical 
baseline were really given to the contractor as opposed to being 
held within the program office. 

Senator Carper. This is sort of the inverse of the situation that 
I think Mr. Jarrett described. 

Mr. Grasso. Exactly. And we find that to be the case quite often. 
When you don’t have a strong program office, a lot gets transferred 
to the contractor and you end up with a lot more dynamics in the 
program. 

Senator Carper. All right. Let me just ask, do you all have any 
closing thoughts that you would like to give us? I like to talk about 
take-aways, and it is impossible for me to remember everything 
that is said here. Even our staff, as smart as they are, it is impos- 
sible to remember everything. But just some of the key take-aways 
for us from what you have heard from other witnesses and maybe 
some points that you have brought up that you would like to reem- 
phasize as we prepare to conclude. 

Is it Secretary Jarrett? Do they call you Secretary? Mr. Sec- 
retary. 

Mr. Jarrett. You may call me anything you wish. 

Senator Carper. No, but seriously, aren’t you a cabinet sec- 
retary. 

Mr. Jarrett. I am. 

Senator Carper. Then I shouldn’t call you Mr. Jarrett. I should 
call you Mr. Secretary. Do you want to take the first shot at that? 

Mr. Jarrett. I guess the only thoughts I had, and it kind of goes 
back to your earlier question about comments from OMB, and that 
is that reports and all the things that they put out are good and 
I guess they are a requirement here, but in fact, I have always be- 
lieved that reports are only as good as how you use them to actu- 
ally make a difference and to change things. 

What I guess I don’t see in a lot of ways is how those reports 
are being utilized to actually begin to change the process. It just 
kind of has become embedded in the reports, and in fact, again, if 
you are not using them, then why even produce them. We spend 
less time on reports and more on the performance of the projects 
themselves. 

The second would be I see a lot of red. Well, we actually end up 
with red on some of our projects, too. The difference is, just because 
it is red doesn’t mean it is ready to go and tank. It just means, at 
least in our case, that it is something that needs a lot of extra look- 
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see and effort on it to get it back to yellow or to green, where it 
ultimately needs to be. Projects go in and out of red, green, and 
yellow all of the time. It depends on the complexity. So I think 
maybe red is nice, but I think there is this perception that red is 
bad. Yes, it is not good, but it is not necessarily the end of all 
things. 

The final thing is that in the issue about information back to 
Congress, I was telling some folks earlier, kind of the mindset 
which they take, and having come from the private sector and then 
Delaware Government for the last 7 Vz years, I am by no means an 
expert in the dynamics of Washington, but we took a very simple 
approach with our legislature which was we will notify them and 
show them this process on everything, painful as it may be on occa- 
sion. I have had some painful discussions with Senators and Rep- 
resentatives one-on-one and even in groups. 

Over time, what it has built, though, is the fact that they have 
come to understand the complexities with projects. They have be- 
come educated that it is not as easy as people think it is and that 
there are a lot of bumps in the road. So now what we find is that 
they have a far better understanding of that and we have a lot 
more productive dialogue back and forth as we work through those 
particular projects than we did when we started 7 years ago. 

So I would recommend to them to take the opposite approach, 
which is to provide that information almost as much as they can 
and not be concerned about the fact of whether it shames somebody 
or not because I think as you said earlier, I think sometimes that 
is not necessarily a bad thing. 

Senator Carper. All right. Thank you. Dr. Brown. 

Mr. Brown. I have three things that might be take-aways. One 
is to actually have Congress require visibility at agencies and at 
OMB. 

The second is to have Congress require the agencies to maintain 
and use effective critical practices. 

And the third is for Congress to maintain its involvement. 

Senator Carper. All right. Expand just a little bit on the second 
one. 

Mr. Brown. On maintaining critical practices, that is to have a 
real program management and oversight capability at each agency. 
What happens is there is so much focus on the Exhibit 300s and 
the planning of programs that their execution and management go 
almost unnoticed, and that is where the troubles begin. It has been 
said that the plan doesn’t survive the first battle, and so, too, with 
the plans for our programs. Things change and are very dynamic. 
Requirements change all the time and that is why, as Mr. Jarrett 
said, change management is very important. 

Agencies tend to ignore change management. They tend to ignore 
a very important thing called risk management to identify risks. 
They don’t pay a lot of attention to what is going on in programs 
in terms of incentivizing the contractor to do the kinds of things 
that are fundamental, to examine what the fundamental capabili- 
ties that are needed by government are and not be absolutely con- 
trolled by requirements, to ignore the kinds of things that allow 
you to have a real confidence in your way of understanding what 
is going on. 
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Instead of just requiring earned value as your sole indicator, you 
could look at real products being developed. You want to have prod- 
ucts being developed all the time, weekly or monthly deliveries to 
the government, and to have assurance that those products are 
being properly integrated together. I have probably given you more 
than you want. 

Senator Carper. All right. Just enough. Thank you. Mr. Grasso. 

Mr. Grasso. I guess I would start by saying this is hard. I have 
seen good organizations fail at IT projects, and I would start by 
simply saying that we all have A teams and B teams. We need a 
lot more A teams, which means investment in people and the proc- 
esses to do this job right. 

The teams need to feel ownership, if you will, in the programs 
and need to have the authorities to do the right things. I once used 
this analogy. As a homeowner, I may make some decisions to make 
some investments in my home. This year, I am going to put $3,000 
into a sprinkler system in my home. Well, as I proceed to do that, 
I spring a leak in my roof. If this were the government, I wouldn’t 
be able to fix that roof because I don’t have the right kind of money 
to fix that roof and I would continue to fix my sprinkler system. 
I don’t have the ownership of that portfolio, if you will, to treat my 
home as a whole complex suite of things. And the ability for a pro- 
gram manager to be able to manage a portfolio as opposed to an 
individual activity will allow him to manage some of those risks 
and make the necessary shifts as is appropriate. 

I mentioned earlier in my testimony about contingencies and 
funding contingencies. A risk management plan is absolutely crit- 
ical to any program, but a risk management plan must have action- 
able alternatives. If the alternatives are not actionable, then we 
will just continue to mount on the risks. 

So I think the key points that I would make is we must continue 
to invest in the people and the organizations. As I mentioned ear- 
lier, the Highly Qualified Experts Program and the Critical Pay 
Program in IRS are good examples of investment. It sounds like 
Delaware has been able to do that. We must be able to manage 
these risks and empower our program managers with strong teams 
to make the right decisions and to have alternatives and choices as 
they see the technologies change and some of those risks surface. 

Senator Carper. Good. Thank you. Well, in closing, I really want 
to commend our staffs for being smart enough to invite you to come 
and testify. The way it works here, you have Democrats and Re- 
publicans who submit ideas and come up with ideas and ultimately 
we end up with a witness list. But this has been an especially help- 
ful and beneficial panel of witnesses for us. 

One of the things that you are especially good at is explaining 
things in terms that even I can almost understand in most cases, 
and these are not easily understood concepts. That is a great gift. 

Thanks a lot for making time in your schedules to be here with 
us today, for preparing for this hearing, and for giving us not just 
a lot of food for thought, but, I think, a pretty good road map to 
follow to better ensure that we are getting our money’s worth out 
of these IT projects that are going on throughout the government, 
actually around the world, and for all the taxpayers who are put- 
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ting their money into this. Thank you for them, too. Keep up the 
good work that you are doing and much obliged. 

With that, you will probably hear from some of us with some ad- 
ditional questions. We will try to get those out in the next week 
or so, but if you could respond promptly, we would be most grate- 
ful. 

Thank you so much, and with that, this hearing is adjourned. 

[Whereupon, at 1:41 p.m., the Subcommittee was adjourned.] 
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Chairman Carper, Ranking Member Cobum, and Members of the Subcommittee, I 
appreciate the opportunity to appear before you today as the Subcommittee discusses the 
Administration’s progress in tracking, analyzing and evaluating the federal government’s 
information technology (IT) investments. 

This morning, I will provide a short overview of agencies’ progress in implementing 
performance-based management for their major acquisition programs. My remarks will 
primarily address non-IT investments. My colleague, the Honorable Karen Evans, Administrator 
for Electronic Government and Information Technology, will discuss the initiatives the 
Administration is pursuing to effectively manage the government’s IT portfolio. 
Performance-based management requirements 

Title V of the Federal Acquisition Streamlining Act (“FASA V”) requires executive 
agencies to establish cost, schedule and performance goals for all major acquisition programs 
and achieve, on average, 90 percent of those goals. Part 7 of OMB Circular A-l 1 (. Preparation , 
Submission, and Execution of the Budget) provides guidance to implement this practice, known 
as performance-based management. Major acquisitions of new capital assets must be justified in 
terms of agency strategic goals and needs, reflect sound acquisition and capital planning 
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decisions, and include measurable cost, schedule, and performance goals. For ongoing 
acquisition programs, agencies must report progress toward achieving baseline goals, explain 
actual or projected deviations from those goals, and describe actual or planned corrective action 
as needed to achieve baseline goals. 

Additional guidance is provided in OMB’s Capital Programming Guide, a supplement to 
Circular A-l 1 Part 7. OMB revised the guide in 2006 to emphasize the importance of key steps 
in the acquisition planning process, such as needs assessment and alternative analyses. The 
guide also includes expanded coverage on earned value management (EVM) to ensure agencies 
have and use objective “early warning” information to identify projects that may be falling short 
on their goals and make reasoned decisions to support corrective actions. 

The Circular prescribes a form. Exhibit 300, for agencies to provide specific information 
on performance goals and measures, results achieved against goals, acquisition strategies, and 
project management. With respect to project management, Exhibit 300 requires agencies to 
report on the qualifications of the assigned program or project manager, and address the use of 
EVM. 

Progress implementing performance-based management 

For a number of years, government efforts to implement performance-based management 
at civilian agencies have focused heavily on major IT acquisition programs. As Administrator 
Evans will explain, the Administration continues to pursue an aggressive strategy for tracking, 
analyzing and evaluating the risks and results of major capital investments for information 
systems. Most agencies are making reasonable progress in implementing performance-based 
management policies and practices for their major IT investments. 
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Progress implementing performance-based management has been more limited on non-IT 
investments. Currently, there is no government-wide portfolio assessment of non-IT capital asset 
classes, which may take a variety of forms, ranging from construction and real-property 
management to aircraft acquisitions. 

To assess agency progress in implementing performance-based management principles 
and practices, the Office of Federal Procurement Policy (OFPP) directed agencies to provide 
information on their new and ongoing non-IT major acquisition projects. OFPP considered 
whether agencies: (1) have capital planning and investment control (CPIC) policies for major 
acquisitions other than IT; (ii) have cost, schedule, and performance goals for new and ongoing 
projects; (iii) use, or plan to use performance-based management systems (PBMS) to monitor 
acquisition progress on ongoing programs and new programs, respectively; and (iv) are meeting 
their cost, schedule, and performance goals. 

A number of agencies could not demonstrate that they were meeting all of these criteria. 
CPIC policies are often not as well established as they are for IT investments. Performance- 
based management systems are not always available to track cost, schedule, and performance. 

As a result, some agencies could not provide sufficient information to demonstrate that cost, 
schedule, and performance goals are being met. 

We can and must do better. OMB is taking steps to strengthen the application of 
performance-based management to non-IT investments. These steps include the following: 

1. Increased collaboration. In late 2007, the Chief Acquisition Officers Council 
(CAOC) established the Project Management Working Group (PMWG) to work with OMB in 
achieving the consistent implementation and use of capital planning and project management 
principles and practices to improve outcomes on federal projects, both for IT and non-IT. 
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Among other things, the PMWG will serve as an inter-agency focal point to help evaluate the 
appropriate application of performance-based management — including tools, models and 
metrics — to different types of major non-IT investments, such as construction and aircraft. 

2. Mitigation of high risk in major acquisitions. Since the early 1990s, the Department 
of Defense (DoD), the Department of Energy (DOE), and the National Aeronautics and Space 
Administration (NASA) have been listed on the High-Risk List established by the Government 
Accountability Office (GAO) for significant vulnerabilities in contract management, including 
the management of major acquisitions. In 2007, OMB partnered with each of these agencies and 
the GAO to collaborate in the development of corrective action plans (CAPs). Each plan will 
identify a clear definition of success tied to successful attainment of cost, schedule, and 
performance goals. Plans are to include outcome and process goals, metrics on cost and 
schedule containment, and corrective steps to meet each goal. Goals and metrics will be shared 
among the agencies so they may build on each other’s efforts to mitigate risk and maximize 
return on investment. DOE’s CAP was finalized earlier this month. We expect NASA and DoD 
to finalize their CAPs by the fall. 

3. Strengthened workforce competencies in program and project management. OFPP, in 
conjunction with the Federal Acquisition Institute and more than twelve federal agencies, 
developed a federal acquisition certification program for program and project managers to 
standardize training and experience requirements for the civilian agencies. The certification 
program promotes development of competencies that are critical to successful performance- 
based management, such as requirements analysis, cost estimating, financial management, risk 
management, and quality assurance. Certification is required for program and project managers 
that are assigned to major acquisitions. These managers will be expected to have at least four 
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years of program and project management experience on federal projects and programs, 
including managing and evaluating agency acquisition investment performance, developing and 
managing a program budget, building and presenting a successful business case, reporting 
program results, strategic planning, and high-level communication with internal and external 
stakeholders. 

4. Improved assessment of internal controls. This past spring, OFPP issued guidance to 
standardize the approach agencies use to assess their acquisition activities. The guidance 
requires agencies to integrate their assessments with other agency internal control reviews 
established under Circular A- 123 (Management’s Responsibility for Internal Control). The 
guidance, which is modeled on an analytical framework developed by the GAO, is designed to 
ensure reviews address the areas that most significantly influence the effectiveness and 
accountability of the acquisition process - namely, organizational alignment and leadership, 
policies and practices, human capital, and information management and stewardship. The 
guidance includes a series of critical questions to help agencies identify factors that may 
contribute to weaknesses in the planning and execution of major projects. For example, the 
guidance helps agencies assess if they are integrating organizational goals into the capital 
decision-making process, evaluating, selecting, and controlling capital assets using an investment 
approach, and balancing budgetary control and managerial flexibility when funding capital 
projects. If a material weakness with an internal control involving a major acquisition was 
identified during an assessment, it would be addressed during corrective action monitoring and 
reflected on the agency’s statement of assurance. 
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Reports on agency implementation efforts 

Information on major capital investments for IT may be found in Chapter 9 of the 
Analytical Perspectives volume of the President’s Budget and on OMB’s E-Government 
homepage at www.omb.gov. This analysis is published annually. Of particular note, Table 9-1 
of Chapter 9 provides an evaluation of each agency’s effectiveness in managing IT investments 
and E-Govemment processes, along with improvement milestones for the calendar year. The 
information in the Budget is provided to Congress and the public in accordance with section 
5 1 12(c) of the Clinger-Cohen Act. 

Information on agency progress in implementing performance-based management for 
both IT and non-IT investments is included in an OFPP report addressing the FASA V 
requirements. OFPP will partner with the CAOC PMWG to ensure timely collection of 
information and preparation and submission of future reports to Congress in accordance with 
FASA V. 

Conclusion 

The Administration strongly supports use of performance-based management for its 
major acquisition programs and remains committed to its effective implementation. This 
discipline helps agencies identify and mitigate cost overruns, schedule delays, and performance 
shortfalls. We will also continue to rely on those additional tools that have helped us achieve 
success - including the management watch list, the high-risk list, EVM and the scorecard -- and 
adopt new tools, as necessary, to ensure taxpayer dollars are invested wisely and managed 
responsibly. 

This concludes my prepared remarks. I am happy to answer any questions you might 

have. 
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Good morning, Mr. Chairman and Members of the Committee. My remarks will focus 
on the Administration’s strategy and continued progress in performing oversight of 
agency capital investments in information technology (IT). Specifically, I will address 
the topic of the Office of Management and Budget’s (OMB’s) ability to effectively 
analyze, track, and evaluate agency major capital IT investments. 

The President’s budget request for FY 2009 includes approximately $71 billion for IT 
and associated support services government-wide. Though this amount does not 
represent a large percentage of the total President’s budget request, the Office of 
Management and Budget (OMB) focuses considerable attention on IT-related budgetary 
issues, program planning and program execution given the cross-cutting nature of 
information systems management as well as the potential impact of investments in IT on 
an agency’s ability to effectively deliver programs and services to the citizens of the 
United States. 

Of the $71 billion in the President’s Budget request for 2009, $22 billion, or 31%, 
represents proposed funding for development, modernization, or enhancement of new or 
existing information systems, infrastructure or services. Projected expenditure on 
existing systems operations and maintenance is $49 billion or 69 percent. 

Capital Planning and Investment Control 

As I have discussed in previous testimony on the topic of OMB’s oversight of IT 
investments, Public Law 104-106, the Clinger-Cohen Act of 1996 established processes 
for executive agencies to analyze, track, and evaluate the risks and results of major 
capital investments for information systems. 

The operative means by which OMB evaluates agency capital asset plans and their 
associated budgetary requests is a business case, or exhibit 300, for individual 
investments. Agencies develop and submit business cases with their annual budget 
request to OMB, and we in turn evaluate each business case in terms of its ability to 
support a given investment proposal, including factors such as alignment with the IT 
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architecture, planned performance improvement goals, cost/benefit analyses, and 
eliminating costly duplicative and outdated systems. 

OMB also evaluates the capability to manage the investment as demonstrated in the 
business case and planning process, including factors such as having a qualified project 
manager, acquisition planning, systems security, risk management, and use of Earned 
Value Management (EVM) to track and manage cost and schedule goals. 

As OMB prepares a budget to sustain current services for Fiscal Year 2010, due to the 
impending change in Administrations, agencies are required to submit updates to their IT 
business cases and non-major systems spending projections by September 8, 2008, to 
allow continuity of oversight and review of IT capital investments in order to prepare for 
the transition period. 

The Management Watch List and High Risk List 

Last year, I came before this committee to explain two specific tools OMB uses to track 
and review agency IT capital investments and high stake and special interest investments, 
the Management Watch List and the High Risk List. I am pleased to report that we have 
since expanded upon our criteria for evaluating investments, and improved upon the 
transparency of how investments and projects are placed on either or both lists. 

We added three overall evaluation criteria affecting all business cases in an agency’s 
investment portfolio to better capture and communicate the impact and importance of 
fundamental IT project management and planning capabilities in the areas of information 
security, privacy and cost, schedule and performance management. Now, if an agency 
has not demonstrated effective processes for systems certification and accreditation and 
performing privacy impact assessments, as evidenced by their Inspector General’s 
evaluation of these processes in the agency’s annual Federal Information Security 
Management Act (FISMA) report 

(http://www.whitehouse.gov/omb/inforeg/reports/2007_fisma_report.pdf), all 
investments within the portfolio are placed on the Management Watch List until those 
core competencies are adequately performed. Likewise, if an agency has failed to 
achieve an average of seventy percent of the cost, schedule, and performance goals for 
acquisitions within the investment portfolio, as evidenced by their performance on the E- 
Govemment scorecard of the President’s Management Agenda (which I will discuss in 
more detail shortly), all investments within the portfolio are placed on the Management 
Watch List until the agency remediates this fundamental investment management risk. 

In terms of increasing transparency, during the preparation of the FY 2009 President’s 
Budget, OMB provided specific feedback to agencies on a business case-by business case 
basis as to where they fell short of the evaluation criteria, published the section-by- 
section evaluation results for each business case on the Management Watch List, and 
published the criteria OMB uses in assigning scores for each section of the business case. 
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We also improved the High Risk List by expanding the information included for each 
project to include the reason(s) why the project is deemed high risk in nature or 
execution, and included this additional detail in the published version for the third quarter 
of FY 2008. 

The Management Watch List and the High Risk List are reported as part of Chapter 9 of 
the Analytical Perspectives Chapter of the Budget, and are thereafter published quarterly 
until the next budget year. 

Projects on the Management Watch List and High Risk List FY 2006, FY 2007, and as of Third Quarter FY 
2008 















Date of Publication 








ESI 



Major IT Investments 

857 

861 

840 

833 

810 

1801 

Management Watch List Investments 

263 

84 

346 

134 

585 

352 








High Risk Projects 

N/A 

234 

477 

603 

601 

477 


Performance Measures for Acceptable IT Business Cases and 
Cost/Schedule/Performance in the President’s Management Agenda Scorecard 

OMB oversees the agencies’ activities under the President’s Management Agenda and its 
associated quarterly reporting process. The Expanding Electronic Government Scorecard 
includes standards for success for the full complement of IT management practices, 
including enterprise architecture, capital planning and investment management, managing 
cost, schedule and performance, information systems security and privacy, and 
implementation of government-wide E-Gov shared service systems and Line of Business 
initiatives. 

Agency scores are posted quarterly at http://results.gov/agenda/scorecard.html . 

The percentage of acceptable business cases within an agency investment portfolio is an 
explicit criterion on the E-Gov Scorecard. Green performance requires an agency not to 
have any business cases on the Management Watch List. Yellow performance requires 
less than half of the business cases in the investment portfolio to be on the Management 
Watch List. 

Starting in Fiscal Year 2008, if an agency’s entire portfolio is on the Management Watch 
List due to the addition of the three aforementioned cross-business case evaluation 
criteria for systems security, privacy impact assessments, and cost, schedule and 
performance management, the agency’s E-Gov Scorecard status is downgraded to red in 
the third quarter of the fiscal year. 
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Cost, schedule aud Performance Management 

The exhibit 300 includes information on program cost and schedule milestones. 
Investments in development, modernization, and enhancement are required to report an 
original and current baseline upon entering into the systems development phase of the 
systems development lifecycle. Also, for projects requiring use of EVM, agencies report 
if cost or schedule variance is within 10%, and if the project has been rebaselined within 
the last fiscal year. 

Green performance for cost, schedule and performance management on the E-Gov 
Scorecard requires a demonstration of management capabilities and sustained 
performance as evidenced by a composite cost and schedule variance within 10% of 
goals for development investments. 

We also collect and review EVM data quarterly from those agencies that are green for 
cost, schedule and performance on the E-Gov Scorecard, as well as from agencies 
expecting to demonstrate green performance in the near term. 

GAO’s recent report on project rebaselining points out weaknesses in agency policies in 
terms of specifying all of the elements of rebaselining according to best practices. We 
acknowledge more should be done to address this in the future. In particular, OMB needs 
to clarify expectations for when an original baseline should be established, and elaborate 
upon the process steps for agencies to submit to and receive feedback from OMB on 
rebaseline requests. 

Though the number of project rebaselines is admittedly high, the practice is not always 
indicative of project management or execution failure. As the GAO report indicates, 

99% of the rebaselines for the sample projects in their survey cited changes in either 
requirements and scope or funding stream as the underlying reason for the rebaseline. 
There is inherent difficulty in projecting and predicting, or failure to accurately project 
and predict, the various internal and external factors and events that can impact an IT 
investment throughout its intended development life cycle. 

To mitigate planning risks, agencies should be using an Integrated Baseline Review 
(IBR) to finalize a project baseline, and to validate the information for a rebaseline where 
appropriate. OMB policy directs agencies to perform Integrated Baseline Reviews 
(IBRs) to finalize the cost, schedule and performance goals of the project. IBRs can be 
conducted pre or post- contract award for systems acquisitions. 

If agencies establish the performance measurement baseline prior to conducting an IBR, 
the likelihood of the baseline having enduring validity is lower. Likewise, if an agency 
establishes detailed original cost and schedule goals via the budget formulation process, a 
forward looking planning activity of two years or more, they may be setting themselves 
up for failure when conditions change at the time of actual contract award and project 
execution. The original baseline should be established at the onset of systems 
development and via a rigorous IBR process. 
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Corrective rebaseiines can and do happen when a project is failing to meet goals due to 
mismanagement. Our shared goal should be to address the causes of project failure, 
before a corrective rebaseline is necessary, rather than focus on the procedural aspects of 
the rebaselining process itself. 

Solutions for Improving Management and Oversight of IT Investments 

OMB, Congress, and agencies must work collaboratively to address weaknesses in IT 
program and project performance — we are all vested in a common interest: delivering 
results for the American people. How can we best do this going forward? I believe 
OMB has the foundational processes in place to perform program and project investment 
oversight at the macro-level. These core processes, capital planning, architectural 
alignment, the Management Watch List, the High Risk List, and the E-Gov scorecard 
have been, and can continue to be, further enhanced and incrementally improved to 
provide better visibility into program performance before a project is in trouble so deep it 
is irreparable. 

One improvement for OMB would be to leverage the efforts of the Budget Formulation 
and Execution Line of Business and to ensure the use of the analytical tools and 
collaboration environment to improve our own information management capabilities 
supporting the management and oversight regarding IT investments and projects. The 
various data sets collected by OMB from agencies, can and should be better integrated 
into a more comprehensive knowledge base. We are working closely with this Line of 
Business to improve our ability to integrate the information. 

I have brought a display of an example of how we could better integrate and expand upon 
information on agency IT investments. This is one potential snap-shot of project 
performance at a point in time. We would like to work with Congress to improve 
transparency and ease of use of this information. With the transparency into 
departments’ and agencies’ performance, will come improved accountability and results 
which the American taxpayers’ deserve. 

Conclusion 

Thank you for this opportunity to discuss the Administration’s strategy and progress to 
date in tracking and evaluating IT investments and projects performance. With this 
foundation in place, we can continue to work together to achieve the outcome we both 
desire which is successful implementation of information technology for program and 
mission results. We have accomplished a lot in the last eight years, and there is more we 
can continue to do this very important area. 
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OMB has taken steps to improve the identification of the Management Watch 
List and high risk projects since GAO testified last September, including 
publicly disclosing reasons for placement on the Management Watch list and 
clarifying high risk project criteria. However, more needs to be done by both 
OMB and the agencies to address recommendations GAO has previously made 
to improve the planning, management, and oversight of poorly planned and 
performing projects so that potentially billions in taxpayer dollars are not 
wasted. 

In its rebaselining review, GAO reports that 48 percent of the federal 
government’s major IT projects have been rebaselined for several reasons, 
including changes in project goals and changes in funding. Of those 
rebaselined projects, 51 percent were rebaselined at least twice and about 11 
percent were rebaselined 4 times or more. In addition, while the major 
agencies have all established rebaselining policies, these policies are not 
comprehensive. Specifically, none of die policies were fully consistent with 
best practices, including describing a process for developing a new baseline 
and requiring the validation of the new baseline. Agencies’ policies varied in 
part because OMB has not issued guidance specifying what elements these 
policies are to include. In its report, GAO makes recommendations to OMB to 
issue guidance for rebaselining policies and to the major agencies to develop 
comprehensive rebaselining policies that address identified weaknesses. 


What GAO Found 

OMB and federal agencies have identified approximately 413 IT projects — 
totaling at least $25.2 billion in expenditures for fiscal year 2008— -as being 
poorly planned, poorly performing, or both. Specifically, through the 
Management Watch List process, OMB determined that 352 projects (totaling 
about $23.4 billion) are poorly planned, in addition, agencies reported that 87 
of their high risk projects (totaling about $4.8 billion) were poorly perfonning. 
Twenty-six projects (totaling about $3 billion) are considered both poorly 
planned and poorly perfonning. 
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Mr. Chairman and Members of the Subcommittee: 

I am pleased to be here today to discuss the federal government’s 
processes for improving the management of information technology 
(IT) investments. As you know, billions of taxpayer dollars are spent 
on these projects each year. This number is expected to reach $71 
billion for fiscal year 2009. Given the size of these investments and 
the criticality of many of the systems to the health, economy, and 
security of the nation, it is important that they be effectively 
managed. 

To this end, the Office of Management and Budget (OMB), which 
plays a key role in overseeing the federal government’s IT 
investments, identifies major projects that are poorly planned by 
placing them on a Management Watch List and requires agencies to 
identify high risk projects that are performing poorly. Having 
accurate and transparent project cost and schedule information is 
also essential to effective oversight. At times, changes to this 
information — called a rebaselining — are made to reflect changed 
development circumstances. These changes can be done for valid 
reasons, but can also be used to mask cost overruns and schedule 
delays. 

We have testified on the Management Watch List and high risk 
projects for the past 2 years, highlighting the number and dollar 
value of the projects identified as poorly planned and/ or poorly 
performing.' You asked us to (1) provide an update on OMB’s 
Management Watch List and list of high risk projects, (2) identify 
OMB’s efforts to improve the identification and oversight of these 
projects, and (3) summarize our IT project rebaselining report, 
which is being released today. ! In preparing this testimony, we 


' GAO, Information Technology: Further Improvements Needed to Identify and Oversee 
Poorly Planned and Performing Projects, GAO-07-1211T (Washington, D.C.: Sept 20, 2007); 
GAO, Information Technology: Improvements Needed to More Accurately Identify and 
Better Oversee Risky Projects Totaling Billions of Dollars, GAO-06-1099T (Washington, 
D.C.: Sept 7, 2006) 

2 GAO, Information Technology: Agencies Need to Establish Comprehensive Policies to 
Address Changes to Projects’ Cost, Schedule, and Performance Goals, GAQ-08-925 
(Washington, D.C.: July 31, 2008). 
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analyzed the current Management Watch List and high risk project 
information and reviewed recent actions taken by OMB to better 
identify and oversee these projects. In completing our rebaselining 
review we surveyed the managers of a random sample of 180 
projects selected from the 778 major IT projects the 24 major 
agencies plan to invest in during fiscal year 2008 and compared 
agencies’ rebaselining policies to best practices identified in our 
Cost Assessment Guide. We performed our work in accordance with 
generally accepted government auditing standards. Those standards 
require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings 
and conclusions based on our audit objectives, We believe that the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 


Results in Brief 


OMB and federal agencies have identified approximately 413 IT 
projects — totaling at least $25.2 billion in expenditures for fiscal 
year 2008 — as being poorly planned, poorly performing, or both. 
Specifically, through the Management Watch List process, OMB 
determined that 352 projects (totaling about $23.4 billion) are poorly 
planned. In addition, agencies reported that 87 of their high risk 
projects (totaling about $4.8 billion) were poorly performing. 
Twenty-six projects (totaling about $3 billion) are considered both 
poorly planned and poorly performing. 

OMB has taken steps to improve the identification of the 
Management Watch List and high risk projects since GAO testified 
last September, including publicly disclosing reasons for inclusion 
on the Management Watch List and clarifying high risk project 
criteria. However, more needs to be done by both OMB and the 
agencies to address recommendations we have previously made to 
improve the planning, management, and oversight of the poorly 
planned and poorly performing projects so that potentially billions 
in taxpayer dollars are not wasted. For example, OMB has yet to 
pubhcly disclose the deficiencies (i.e., performance shortfalls) 
associated with high risk projects, and agencies still need to take 
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actions to address recommendations we have previously made to 
improve their investment management practices. 

In our rebaselining review, we project that 48 percent of the federal 
government’s mqjor IT projects have been rebaselined for several 
reasons, including changes in project goals and changes in funding. 
Of those rebaselined projects, 51 percent were rebaselined at least 
twice and about 1 1 percent were rebaselined 4 times or more. In 
addition, while the mqjor agencies had all established rebaselining 
policies, these policies were not comprehensive. Specifically, none 
of the policies were fully consistent with best practices, including 
describing a process for developing a new baseline and requiring the 
validation of the new baseline. Agencies' policies varied in part 
because OMB has not issued guidance specifying what elements 
these policies are to include. In our report, we are making 
recommendations to OMB to issue guidance for rebaselining 
policies and to the major agencies to develop comprehensive 
rebaselining policies that address identified weaknesses. 


Background 

Each year, OMB and federal agencies work together to determine 
how much the government plans to spend on IT projects and how 
these funds are to be allocated. Federal IT spending has risen to an 
estimated $71 billion for fiscal year 2009. 

OMB plays a key role in overseeing the implementation and 
management of federal IT investments. To improve this oversight, 
Congress enacted the Clinger-Cohen Act in 1996, expanding the 
responsibilities delegated to OMB and agencies under the 
Paperwork Reduction Act.’ Among other things, Clinger-Cohen 
requires agencies to better link their IT planning and investment 
decisions to program missions and goals and to implement and 
enforce IT management policies, procedures, standards, and 
guidelines. The act also requires that agencies engage in capital 


3 44 U.S.C. § 3604(h) & 3506(1>). 
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planning and performance and results-based management.* OMB’s 
responsibilities under the act include establishing processes to 
analyze, track, and evaluate the risks and results of major capital 
investments in information systems made by executive agencies. 
OMB must also report to Congress on the net program performance 
benefits achieved as a result of these investments, 5 

In response to the Clinger-Cohen Act and other statutes, OMB 
developed policy for the planning, budgeting, acquisition, and 
management of federal capital assets. This policy is set forth in OMB 
Circular A-ll (section 300) and in OMB’s Capital Programming 
Guide (supplement to Part 7 of Circular A-l 1), which direct agencies 
to develop, implement, and use a capital programming process to 
build their capital asset portfolios. Among other things, OMB’s 
Capital Programming Guide directs agencies to 

• evaluate and select capita] asset investments that will support core 
mission functions and demonstrate projected returns on investment 
that are clearly equal to or better than alternative uses of available 
public resources; 

• institute performance measures and management processes that 
monitor actual performance and compare it to planned results; and 

• establish oversight mechanisms that require periodic review of 
operational capital assets to determine if mission requirements have 
changed and whether the asset continues to fulfill those 
requirements and deliver its intended benefits. 

To further support the implementation of IT capital planning 
practices as required by statute and directed in OMB’s Capital 
Programming Guide, we have developed an IT investment 
management framework 6 that agencies can use in developing a 
stable and effective capital planning process. It is a tool that can be 
used to determine both the status of an agency’s current IT 


'40U.S.C.5 11312 & 113 13. 
s 40U.S.C.§ 11302 &11303. 

fi GAO, Information Technology Investment Management: AFramework for Assessing and 
Improving Process Maturity, GAO-04-394 G (Washington, D.C.: March 2004). 
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investment management capabilities and the additional steps that 
are needed to establish more effective processes. Mature and 
effective management of IT investments can vastly improve 
government performance and accountability, while poor 
management can result in wasteful spending and lost opportunities 
for improving delivery of services to the public. 


Prior Reviews on Federal IT Investment Management Have Identified Weaknesses 

We have previously reported that the federal government faces 
enduring challenges in effectively managing IT investments. For 
example, in January 2004, we reported on the mixed results of 
federal agencies’ use of IT investment management practices.' 
Specifically, we reported that, although most of the agencies had IT 
investment boards responsible for defining and implementing the 
agencies' investment management processes, they did not always 
have important mechanisms in place for these boards to effectively 
control investments, including decision-making rules for project 
oversight, early warning mechanisms, and/or requirements that 
corrective actions for underperforming projects be agreed upon and 
tracked. Accordingly, we made several recommendations to 
agencies to improve their practices. 

In previous work using our investment management framework, we 
reported that the use of IT investment management practices by 
agencies was mixed. For example, a few agencies that have 
followed the framework in implementing capital planning processes 
have made significant improvements.* In contrast, however, we and 
others have continued to identify weaknesses at agencies in many 
areas, including immature management processes to support both 
the selection and oversight of mqjor IT investments and the 
measurement of actual versus expected performance in meeting 


7 GAO, Information Technology Management: Govemmentwide Strategic Planning, 
Performance Measurement, and Investment Management Can Be Further Improved, GAO- 
04-49 (Washington, D.C.: Jan. 12, 2004). 

8 These agencies include the Departments of Agriculture, Commerce, and the Interior. 
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established performance measures. 9 For example, in 2007, we 
reported that the Department of Homeland Security and the 
Department of the Treasury did not have the processes in place to 
effectively select and oversee their mqjor investments.” 


OMB’s Management Watch List Is Intended to Correct Project Weaknesses and Business 
Case Deficiencies 

To help ensure that investments of public resources are justified and 
that public resources are wisely invested, OMB began using its 
Management Watch List in the President’s fiscal year 2004 budget 
request as a means to oversee the justification for and planning of 
agencies’ IT investments. This list was derived based on a detailed 
review of each investment’s Capital Asset Plan and Business Case, 
also known as the exhibit 300. 

The exhibit 300 is a reporting mechanism intended to enable an 
agency to demonstrate to its own management, as well as to OMB, 
that a major project is well-planned in that it has employed the 
disciplines of good project management; developed a strong 
business case for the investment; and met other Administration 
priorities in defining the cost, schedule, and performance goals 
proposed for the investment. 

In April 2005, we reported that OMB analysts evaluated agency 
exhibit 300s by assigning scores to each exhibit 300 based on 
guidance presented in OMB Circular A-l l.“ As described in this 


9 For example, GAO, Information Technology; Treasury Needs to Strengthen Its Investment 
Board Operations and Oversight, GAO-07-865 (Washington, D.C.; Jul. 23, 2007); GAO, 
Information Technology: DHS Needs to Fully Define and Implement Policies and 
Procedures for Effectively Managing Investments, GAO-07-424 (Washington, D.C., Apr. 27, 
2007); GAO, Information Technology: Centers for Medicare & Medicaid Services Needs to 
Establish Critical Investment Management Capabilities, GAO-06-12 (Washington, D.C.: Oct. 
28, 2005); GAO, Information Technology: Departmental Leadership Crucial to Success of 
Investment Reforms at Interior, GAO-03-1028 (Washington, D.C.: Sept. 12, 2003); and GAO, 
United States Postal Service: Opportunities to Strengthen IT Investment Management 
Capabilities GAO-03-3 (Washington, D.C.: Oct. 15, 2002). 

10 GAO-07-424 and GAO-07-865. 

“GAO, Information Technology: OMB Can Make more Effective Use of Its Investment 
Reviews, GAO-05-276 (Washington, D.C., April 15, 2006) 


Page 6 


GAO-08- 10I5T 



58 


circular, the scoring of a business case consisted of individual 
scoring for 10 categories (on a scale from 1 to 5 with 5 being the 
highest), as well as a total composite score of all the categories. The 
10 scoring categories are: 

• support of the President’s Management Agenda. 

• project (investment) management, 

• acquisition strategy, 

• performance information, 

• security, 

• privacy, 

• enterprise architecture, 

• alternatives analysis, 

• risk management, and 

• cost/ schedule/ performance. 

When we reported on the Management Watch List in 2005, projects 
were placed on the Management Watch List if they received low 
scores (3 or less) in the areas of performance goals, performance- 
based management systems, security and privacy or a low 
composite score. 

For the fiscal year 2009 budget, OMB used more stringent criteria. 
Specifically, OMB placed projects on the Management Watch List if 
they had (1) an overall score of 30 or less, (2) a security score of 3 or 
less, or (3) a non-security score of 2 or less. Projects were also 
placed on the Watch List if other sources (such as an Inspector 
General’s Federal Information Security Management Act report 12 or 
the agency’s E-gov President’s Management Agenda scorecard 13 ) 


12 The Federal Information Security Management Act directs federal agencies to conduct 
periodic information security reviews and Inspectors General to perform annual 
independent evaluations of agency programs and systems and report their results to OMB 
and Congress; 44 U.S.C. § 3544(bX5) &3545. 

13 The President’s Management Agenda is a program that was instituted in 2002 to improve 
the management and performance of the federal government It addresses 5 
govemmentwide initiatives, including E-govemment, that agencies are supposed to 
implement to achieve improvements. OMB issues scorecards on a quarterly basis to track 
how well the departments and major agencies are executing these initiatives. 
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indicated that the agency did not have qualified project managers or 
had weaknesses in its implementation of security, privacy, or earned 
value management techniques. 

According to OMB, agencies with projects on the Management 
Watch List are to submit remediation plans addressing the 
weaknesses. Those projects that receive specific follow-up attention 
receive feedback through what is known as the passback process, 
targeted evaluation of remediation plans designed to address 
weaknesses, the apportioning of funds made conditional on 
appropriate remediation plans being in place, and the quarterly e- 
Gov Scorecards. According to OMB, it removes projects from the 
Management Watch List as agencies remediate the weaknesses 
identified with these projects’ business cases. 


OMB’s High Risk Projects Process Is Intended to Correct and Improve Project 
Performance 


As originally defined in OMB Circular A-l 1 and subsequently 
reiterated in an August 2005 memorandum, high risk projects are 
those that require special attention from oversight authorities and 
the highest levels of agency management. These projects are not 
necessarily “at risk” of failure, but may be on the list because of one 
or more of the following four reasons; 

• The agency has not consistently demonstrated the ability to manage 
complex projects. 

• The project has exceptionally high development, operating, or 
maintenance costs, either in absolute terms or as a percentage of 
the agency’s total IT portfolio. 

• The project is being undertaken to correct recognized deficiencies 
in the adequate performance of an essential mission program or 
function of the agency, a component of the agency, or another 
organization. 

• Delay or failure of the project would introduce for the first time 
unacceptable or inadequate performance or failure of an essential 
mission function of the agency, a component of the agency, or 
another organization. 

In 2006, we reported that, to identify high risk projects, staff from 
each agency’s Office of the Chief Information Officer compare the 
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criteria against their current portfolio to determine which projects 
met OMB’s definition. They then submit the list to OMB for review. 
According to OMB and agency officials, after the submission of the 
initial list examiners at OMB work with individual agencies to 
identify or remove projects as appropriate. According to most 
agencies, the final list is then approved by their Chief Information 
Officer." 

This year, OMB clarified/ expanded the high risk project criteria. 
Specifically, in the materials supplementing the President’s Budget 
for fiscal year 2009, OMB listed the following criteria for these 
projects: 

• complex projects; 

• projects with a high degree of political or citizen interest; 

• projects with cross-organizational or agency impact or 
interdependencies with other systems efforts; 

• mqjor systems on the Management Watch List at the conclusion of 
the prior fiscal year that continue to warrant heightened attention 
during project execution; 

• major systems formally designated as an E-Govemment or Line of 
Business Shared Sendee Provider; 

• E-Govemment initiative migration projects that are planned or 
underway (which are removed upon completion); 

• existing or legacy agency systems retiring once their functionality 
has been migrated to a common solution (also removed once 
retired); and 

• program or program management office activities supporting 
government-wide common solutions. 

For the identified high risk projects, Chief Information Officers are 
to assess, confirm, and document projects’ performance. 
Specifically, agencies are required to determine, for each of their 
high risk projects, whether the project was meeting one or more of 
four performance evaluation criteria, which include: 


11 GAO, Information Technology Agencies and OMB Should Strengthen Processes for 
Identifying and Overseeing High Risk Projects, GAQ06-647 (Washington, D.C., June 15, 
2006) 
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• establishing baselines with clear cost, schedule, and performance 
goals; 

• maintaining the project’s cost and schedule variances within 10 
percent; 

• assigning a qualified project manager; and 

• avoiding duplication by leveraging inter-agency and 
govemmentwide investments. 

High risk projects failing to meet any of these four performance 
evaluation criteria are considered as having “performance 
shortfalls.” Agencies are instructed to document these shortfalls 
using a standard template provided by OMB and provide this 
template to oversight authorities on request. Upon submission, 
individual analysts review the quarterly performance reports of 
projects with shortfalls to determine how well the projects are 
progressing and, using other performance data already received, 
whether the actions described in the planned improvement efforts 
are adequate. 


OMB’s Role and Laws and Guidance on IT Project Rebaselining 

At times, a major IT project's cost, schedule, and performance 
goals— known as a baseline — need to be modified to reflect new 
circumstances. While these changes — generally referred to as 
rebaselining — can be done for valid reasons — including, for 
example, changes in a project’s objectives, scope, requirements, or 
funding stream, they can also be used to mask cost overruns and 
schedule delays. The purpose of a rebaselining is to ensure that 
project managers have realistic benchmarks for tracking the status 
of the project 

OMB requires that all proposed changes to baselines be submitted 
to it prior to an agency’s budget request (and that proposed changes 
should not be assumed to be approved). The information OMB 
requires from agencies includes costs and milestones from both the 
initial and current baselines (if the program has been rebaselined). It 
also asks agencies whether the investment was rebaselined during 
the past fiscal year and, if so, if the new baseline was approved by 
the agency head. The Capital Programming Guide s&so notes that 
OMB reviews the reasons for deviation from goals, the 


Page 10 


GAO-08-1015T 



62 


reasonableness of the corrective actions proposed, and the validity 
of increased cost estimates. The guide further states that OMB is to 
consider approving a rebaseline proposal only when the agency has 
provided justification based on an integrated baseline review, ls 
demonstrates that the new goals have a high probability of 
attainability, and shows that the acquisition will still have a benefit- 
cost ratio that justifies continued funding after comparing it with the 
other projects in the portfolio and considering budget limitations. 

Staff from OMB’s Office of E-government and Information 
Technology and the Acting Chief of OMB's Information Policy and 
Technology Branch told us that they review agencies’ earned value 
management policies to determine their compliance with the 
provisions of the Presidential Management Agenda for E- 
govemment. They stated that, in reviewing these policies, they 
determine whether rebaselining is adequately addressed. 

In addition, the Department of Defense (DOD) has statutory 
requirements involving rebaselining. Each mqjor defense acquisition 
program is required by statute to establish an approved program 
baseline before entering into the system development and 
demonstration phase of the acquisition cycle. For such programs, a 
revised baseline is also required for each subsequent milestone 
authorizing entry into the next phase of the acquisition cycle. The 
statute also requires DOD to prescribe regulations addressing the 
content of the baseline; reports of deviations from the baseline; 
procedures for reviewing such deviations within DOD; and 
procedures for submission to and approval by the Secretary of 
Defense of revised baselines. 

We also recently issued a draft GAO Cost Assessment Guide on best 
practices for estimating and managing program costs'* which, among 
other things, discusses considerations in rebaselining programs. For 


ls An integrated baseline review is an evaluation of a program's baseline plan to determine 
whether all program requirements have been addressed, risks have been identified, 
mitigation plaits are in place, and available and planned resources are sufficient to 
complete the work. 

16 GAO, Cost Assessment Guide: Best Practices for Estimating, and Managing Program 
Costs, exposure draft, (Washington, D.C.: July 2007). 
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example, the guide identifies key cost, schedule, project, execution 
risk, and data accuracy indicators that can serve as warning signs 
that a program may need to be rebaselined. The guide also identifies 
best practices that are relevant to rebaselining policies. These 
practices are: (1) describing reasons when a rebaseline is warranted, 
(2) describing the process for developing a new baseline, (3) 
requiring validation of the new baseline, (4) requiring management 
review, and (5) requiring that decisions associated with the 
rebaselining process are documented. 


Hundreds of Projects Totaling Billions of Dollars in Estimated 
Expenditures for Fiscal Year 2009 Are Poorly Planned or Poorly 
Performing 

OMB and federal agencies have identified approximately 413 IT 
projects — totaling at least $25.2 billion in expenditures for fiscal 
year 2009 — as being poorly planned, poorly performing, or both. 
Specifically, hundreds of projects totaling billions of dollars have 
been placed on OMB’s Management, Watch List for fiscal year 2009. 
In addition, projects identified as poorly performing under OMB’s 
high-risk process total about $4.8 billion in estimated expenditures 
for fiscal year 2009. Finally, 26 projects totaling $3 billion have been 
identified as both poorly planned and poorly performing. 

Figure 1 shows the distribution of these projects and their 
associated dollar values. 


Figure 1 : Poorly Planned and Poorly Performing IT Projects (as of July 2008) 
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Hundreds of Projects Totaling Billions of Dollars Were Placed on the Management 
Watch List for Fiscal Year 2009 

Each year, OMB has placed hundreds of projects totaling billions of 
dollars on the Management Watch List. Table 1 provides a historical 
perspective of the number of these projects and their associated 
budgets since OMB started reporting on the Management Watch List 
in the President’s budget request for 2004. The table shows that 
while the number of projects and their associated budgets have 
generally decreased since then, they increased by 239 projects and 
$13 billion dollars for fiscal year 2009, and represent a significant 
percentage of the total budget. 


Table 1 : Major IT Projects on the Management Watch List for Fiscal Years 2004-2009 


Fiscal years 

Major federal IT 
projects 
(associated 
budget in billions) 

Management 
Watch List 
projects 
(associated 
budget in 
billions) 

Percentage of 
federal IT projects 
on Management 
Watch List 
(percentage of 
budget) 

2004 

1400 ($59.0) 

771 ($20.9) 

55% (35%) 

2005 

1200 ($60.0) 

621 ($22.0) 

52% (37%) 

2006 

1087 ($65.0) 

342 ($15.0) 

31% (23%) 

2007 

857 ($64.0) 

263 ($9.9) 

31% (15%) 

2008 

840 ($65.0) 

346 ($14.0) 

41% (22%) 

2009 

810 ($70.7) 

585 ($27.0) 

72% (38%) 


Source- GAO analysis of OMB dew. 


As of July 2008, OMB reported that 352 of the 585 projects, 
representing $23.4 billion, still remained on the Management Watch 
List (see appendix 1 for complete list). Table 2 shows the number of 
projects each agency has on the watch list as of July 2008. 


Table 2: Number of Major IT Projects on Watch List by Agency (as of July 2008) 


Agency 

Number of Projects 

Percentage of Agencies’ 
Major Projects 

Corps of Engineers 

10 

100% 

Department of Agriculture 

37 

100% 

Department of Commerce 

61 

100% 
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Department of Defense 

63 

100% 

Department of Education 

11 

39% 

Department of Health and 

Human Services 

29 

43% 

Department of Homeland 

Security 

28 

39% 

Department of Interior 

50 

100% 

Department of State 

1 

5% 

Department of 

Transportation 

2 

4% 

Department of Treasury 

4 

7% 

Department of Veteran 

Affairs 

40 

100% 

Nuclear Regulatory 

Commission 

15 

100% 

Office of Personnel 

Management 

1 

100% 

Total 

352 

44% 


Source OMB data 


According to OMB’s evaluation of the exhibit 300s, investments 
were placed on the watch list primarily because of weaknesses in 
the way they addressed (1) cost, schedule, and performance, (2) 
security, (3) privacy, and (4) acquisition strategy. Figure 2 illustrates 
the frequency of these reasons for the projects that remained on list 
as of July 2008. Appendix II provides additional detail by agency, 
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Figure 2; Frequency of Reasons for inclusion on the Management Watch List as of July 2008 

Frequency of reasons 
90 



*Note: Frequency does not add up to 352 because projects could be placed on the Management 
Watch List for multiple reasons. 


in addition, according lo OMB, thirty-two of these projects have 
been on the Management Watch List since fiscal year 2006. The 
departments of Veterans Affairs and Commerce have had the most 
projects on the list since then: 14 and 7, respectively. Table 3 
identifies the 32 projects that have been on the Management Watch 
List since fiscal year 2006. 


Table 3: Projects on the Management Watch List since fiscal year 2006 

Agency 

Project 

Department of Agriculture 

Human Resources Line of Business: Service Center 

Department of Commerce 

Financial Management IT Operations 


Weather and Climate Computing infrastructure 


Services 

Consolidated IT Infrastructure 

National Air Quality Forecast Capability 
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Department of Education 


Department of the Interior 

Department of Transportation 
Department of the Treasury 
Department of Veterans Affairs 



Corps of Engineers 


Office of Personnel Management 


Project 

Next Generation Weather Radar System Product 

Improvement 

National Weather Service Telecommunication 
Gateway System 


National Weather Service Regions & Field 

Common Services for Borrowers-legacy 


Capstone Facility Management System 

Consolidated Infrastructure, Automation, Telecomm 

IT Combined Infrastructure 

Enterprise IT infrastructure Optimization Initiative 

Health Administration Center IT Operations 

Health Data Repository 

Benefits Delivery Network Maintenance and 

Operations 

Decision Support System 

VistA-Legacy 

Federal Health Information Exchange 

Enrollment Operations and Maintenance 

VistA Imaging 

Allocation Resource Center 

Learning Management System 

Medical and Prosthetic Research Operations 

IT Infrastructure-2009 

Program Integrity/Data Management 

Benefits Support Services 

Financial Management System 


Automated Personal Property Management Systems 


Project Management Information System II 


Corps Water Management System 


Enterprise Infrastructure 


Poorly Performing Projects Total About $4.8 Billion in Estimated Expenditures for 
Fiscal Year 2009 

As of June 2008, the 24 major agencies identified 472 IT projects as 
high risk, at least 87 of which had performance shortfalls 
collectively totaling about $4.8 billion in funding requested for fiscal 
year 2009. Table 4 shows that the number of projects increased, 
while the number of projects with shortfalls decreased this year. 
OMB attributes the decrease in projects with performance shortfalls 
to the fact that many of the e-Gov projects have now been 
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implemented and are therefore no longer considered “at-risk.” In 
addition, the Department of Veterans Affairs has not yet provided 
information on its number of projects with shortfalls. 


Table 4: High Risk Projects with Performance Shortfalls (associated budget in 
billions) 

Fiscal years 

Total federal 
IT projects 

High risk 
projects 
(associated 
budget) 

High risk 
projects 
with 
shortfalls 
(associated 
budget) 

Percentage 
of total IT 
projects 
with 
shortfalls 

Percentage 
of total IT 
projects’ 
budget with 
shortfalls 

2007 

857 ($64.0) 

226 ($6.4)* 

79 ($2.2)* 

9% 

3.4% 

2008 

840 ($65.0) 

438 ($14.0) b 

124 ($6.0) 6 

15% 

9% 

2009 

810 ($70.7) 

472 ($14.7) c 

87 ($4.8)** 

10% 

7% 


Source' GAO analysts of OM8 and agency data 


These number and dollar figures are from September 2006. 

" These number and dollar figures are from June 2007. 
c These number and dollar figures are from June 2008 

* These number and dollar figures do not include the Department of Veterans Affairs. 


The majority of projects were not reported to have had performance 
shortfalls. Further, seven agencies — the Department of Housing and 
Urban Development, the Department of State, the National 
Aeronautics and Space Administration, the Nuclear Regulatory 
Commission, the National Science Foundation, the Small Business 
Administration, and the Social Security Administration — reported 
that none of their high risk projects experienced any performance 
shortfalls. Figure 3 illustrates the number of high risk projects by 
agency as of June 2008, with and without shortfalls. 
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Source: OAO analysis of agency data. 

“The Oepartmenl of Veterans Affairs has not yet provided its June 2008 high risk report to GAO. 

Note: One project can hove multiple shortfalls. 

Note: Department of Homeland Security (OHS): Department of Veterans Affairs <VA); Depattmeni of Transportation (DOT); O.S. 
Agency for Internationa' Development (USAID); Social Security Adm frustration (SSA); General Services Administration (GSA); 
Department of Agriculture (USD A): SmaS Business Administration (SBA): Department of Defense (DOD); Environmental Prolection 
Agency (HPA), Nuclear Regulatory Commission (NRC): Office of Personnel Managemeni (OPM): Department of Health and Human 
Services (HHS). Department of Justice (DOJ): National Aeronautics and Space Administration (NASA): National Science Foundation 
(NSF); Department of Housing and Oman Development (HUD): Department of Energy (DOE): Department ot Ubor (DDL); Oopartmenl 
of Interior (OOI) 


Agencies reported cost and schedule variances that, exceeded 10 
percent as the most common shortfall. This is consistent with what 
they reported about a year ago, and the distribution of shortfall 
types is similar to that of last year. Figure 4 illustrates the reported 
number and type of performance shortfalls associated with high risk 
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projects, and appendix III provides additional details of the 
shortfalls associated with each of the poorly performing projects. 


Figure 4: Reporter! Performance Shortfalls of 87 Projects {as of June 2008 ) 


Number of projects 



Cost and schedule Unclear baselines Project manager Duplication 

variance exceed 1 0% not qualified 

Type of High Risk List shortfalls 


Source: GAO analysis of agency data. 


Seventeen high risk projects have experienced performance 
shortfalls for the past four quarters (see figure 5). 
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Figure 5: High Risk Projects with Shortfalls in the Last 4 Quarters Sorted by Funding 
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Sou res: GAO anaty^s of agency data. 

Nota. Department of Homeland Security (OHS); Department of Transportation (DOT) Department ot Justice <OOJ); Environmental 
Protection Agency (EPA); Department of Interior (DOI); Department ot Health and Human Services (HHS); U.S. Agency tor 
International Development (USAID) 


Of these projects, two projects have had shortfalls since the list of 
high risk projects was established in September 2005: 

• DHS’s Customs and Border Patrol Secure Border Initiative 
Network Technology Program, which is expected to provide 
on-scene agents near real-time information on attempted border 
crossings by illegal aliens, terrorists, or smugglers. 

• DHS’s Transportation Security Administration Transportation 
Worker Identification Credentialing, which is to establish a 
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system-wide common secure biometric credential, used by all 
transportation modes, for personnel requiring unescorted 
physical and/or logical access to secure areas of the 
transportation system. 


Several Projects Are Both Poorly Planned and Poorly Performing 

As of July 2008, 26 projects are on both the Management Watch List 
and list of high risk projects with shortfalls, meaning that they are 
both poorly planned and poorly performing. They total about $3 
billion in estimated expenditures for fiscal year 2009. This is an 
increase of 5 projects but a decrease of $1.1 billion from when we 
reported last year. These projects are listed in table 5 below. (Note: 
the project names were taken from OMB’s Management Watch List 
released in July 2008 and matched to those in agencies’ June 2008 
quarterly high risk reports.) 


Table 5: Projects on Both the Management Watch List and the High Risk List with Shortfalls 

Agency 

Investment Name 

Fiscal year 2009 
request 

Department of Homeland 
Security 

NPPD IICP 


15 


Computer Emergency Readiness Team 


109 


Transformation & Systems Consolidation 


19 


Science and Technology Disaster Management E-Gov 


13 


Infrastructure 


1,461 


Homeland Secure Data Network 


48 


USCiS- Transformation 


71 


Immigration - CLAIMS 3.0 


11 


Federal Financial Management System 


30 


Secure Border Initiative net Technology Program 


157 


Crew Vetting 


21 


FAMS Air to Ground Communications & Tactical Information Sharing 


13 


Vessel Logistics System 


4 


Marine Information for Safety and law Enforcement 


13 


NPPD Information Systems Security Line of Business 


3 


eNEMIS 


13 

Department of Interior 

IMARS 


12 

ePIanning 1 


Minerals Management Service - OCS Connect 


6 

Department of Education 

Common Origination and Disbursement 


45 
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Agency 

investment Name 

Fiscal year 2009 
request 


ADvance (Aid Delivery) 


69 

Department of Heath and 
Human Services 

Federal Health Architecture 


4 


Commissioned Corps Force Management Solution 


4 

Department of Treasury 

Treasury-Wide Enterprise Content Management Services 


28 

Department of Agriculture 

Consolidated Infrastructure, Office Automation and Telecommunications 


872 


National Animal Identification System 


5 

Source’ GAO Analysis of OMB and agency data. 


Steps Taken to Improve the Identification of Management Watch List 
and High Risk Projects, But Project Totaling Billions of Dollars Still 
Require Oversight 


OMB has taken steps to improve the identification of the 
Management Watch List and high risk projects since we testified last 
September, including publicly disclosing reasons for placement on 
the Management Watch List, and clarifying high risk project criteria, 
however, more needs to be done by both OMB and the agencies to 
fully address recommendations we have previously made to 
improve the planning, management, and oversight of the poorly 
planned and poorly performing projects so that potentially billions 
in taxpayer dollars are not wasted. 

Management Watch List: In order for OMB to take advantage of 
the potential benefits of using the Management Watch List as a tool 
for analyzing and following up on IT investments on a 
govemmentwide basis, in 2005, we recommended that the agency 
take the following 4 actions: (1) develop a central list of 
Management Watch List projects and their deficiencies; (2) use the 
list as the basis for selecting projects for follow-up and for tracking 
follow-up activities — to guide follow-up, develop specific criteria for 
prioritizing the IT projects included on the list, taking into 
consideration such factors as the relative potential financial and 
program benefits of these IT projects, as well as potential risks; (3) 
analyze the prioritized list to develop govemmentwide and agency 
assessments of the progress and risks of IT investments, identifying 
opportunities for continued improvement; and (4) report to the 
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Congress on progress made in addressing risks of major IT 
investments and management areas needing attention. 

OMB has taken steps to address our recommendations for 
developing a central list of projects and their deficiencies and 
developing govemmentwide and agency assessments. Specifically, 
as previously noted, OMB started issuing a central list of 
Management Watch List projects in September 2006, and publicly 
disclosing these projects’ deficiencies (i.e., the reasons for inclusion 
on the Management Watch List) in April. In addition, OMB 
performed govemmentwide and agency-specific analyses of 
projects’ deficiencies in April and in July of this year which it 
reported to Congress and disclosed publicly. However, OMB needs 
to continue to use the Management Watch List to prioritize the 
projects needing follow up action and to keep reporting to Congress 
on management areas needing attention. 

High Risk Projects: To improve the identification and oversight of 
the high risk projects, in 2006, we recommended, among other 
things, that OMB establish a structured, consistent process to 
update the list of high risk projects on a regular basis, including 
identifying new projects and removing previous ones to ensure the 
ensure the list is current and complete. We also recommended that 
OMB develop a single aggregate list of high risk projects and their 
deficiencies and use that list to report to Congress progress made in 
correcting high risk problems, actions under way, and further 
actions that may be needed. 

OMB took several steps to address these recommendations. As 
previously noted, the agency clarified the high risk project criteria 
this year. It also asked agencies to identify, in their quarterly 
reports, reasons for placement on the list and reasons for removal, 
thereby adding structure and consistency to the process for 
updating the list. In addition, as previously reported, OMB also 
started publicly releasing aggregate lists of the high risk projects in 
September 2006, and has been releasing them on their website on a 
quarterly basis since then. However, OMB has yet to identify the 
deficiencies (i.e., performance shortfalls) associated with the high 
risk projects as we have done in this report (see appendix III). As 
we have stated before, doing so would allow OMB and others to 
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better analyze the reasons projects are poorly performing, take 
corrective actions, and track these projects on a govemmentwide 
basis. Such information would also help to highlight progress made 
by agencies or projects, identify management issues that transcend 
individual agencies, and highlight the root causes of 
govemmentwide issues and trends. 

In addition, as noted earlier, our prior reviews of federal IT 
management practices have identified (and continue to identify) 
weaknesses at agencies that have yet to be addressed. While these 
agencies have taken action to address the many recommendations 
we have made to improve their practices, more needs to be done as 
evidenced by the large number of projects that are still poorly 
planned and poorly performing. 

While the actions taken have resulted in better data on the poorly 
planned and performing projects, additional steps need to be taken 
by both OMB and the agencies to address recommendations we 
have previously made to improve the planning, management, and 
oversight of these projects. These steps include using the 
Management Watch List to prioritize follow up activities. Until these 
additional steps are taken, potentially billion of taxpayer dollars are 
at risk of being wasted. 


About Half of Major IT Projects Have Been Rebaselined Using 
Policies that Are Not Fully Consistent with Best Practices 

Given that cost and schedule variances are the primary reason for 
poorly performing projects, having accurate and transparent cost 
and schedule information is essential to effective oversight. In a 
report being released today, we estimate that about 48 percent” of 
the federal government’s major IT projects have been rebaselined. 
Of those rebaselined projects, 51 percent were rebaselined at least 


”AU percentage estimates based on our survey have 95 percent confidence intervals that 
are within +/- 11 percentage points of the estimate itself. 
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twice and about 11 percent were rebaselined 4 times or more. 18 
These projects are rebaselined for several reasons, including 
changes in project goals and changes in funding. While the major 
agencies have all established rebaselining policies, these policies are 
not comprehensive. Specifically, none of the policies are fully 
consistent with best practices, including describing a process for 
developing a new baseline and requiring the validation of the new 
baseline, identified in our cost assessment guide. Agencies’ policies 
vary in part because OMB has not issued guidance specifying what 
elements these policies are to include. 


About Half of IT Projects Were Rebaselined for Several Reasons 

In our report we project that 48 percent of the major projects 
federal agencies plan to fund in fiscal year 2008 have been 
rebaselined, and about half of those have been rebaselined at least 
twice. Figure 5 summarizes the percentage of projects rebaselined 
and figure 6 summarizes the estimated frequencies of the number of 
times rebaselined major IT projects were rebaselined. 


Figure 5: Estimated Percentage of Major FY2008 Funded IT Projects Rebaselined 



ls We surveyed the managers of a random sample of 180 projects selected from the 778 
major IT projects the 24 major agencies plan to invest in during fiscal year 2008. 
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Figure 6: Estimated Frequency of the Number of Times Rebaselined Projects Were 
Rebaselined 

Percentage of projects 



Times rebaselined 

Soiiice GAO survey or major IT projects 


Table 5 lists the nine projects in our sample that agencies reported 
having been rebaselined four or more times. 19 


Table 5: Projects Rebaselined Four or More Times 


Department 

Project 

Number of times 
rebaselined 

Department of Defense 

Advanced Field Artillery 

Tactical Data System 

4 

Department of Energy 

Licensing Support Network 

4 

Department of Homeland 
Security 

Coast Guard Rescue 21 

4 

Department of Housing and 
Urban Development 

Integrated Human 

Resources and Training 

System 

4 


,9 This lists only the projects in our sample that have been rebaselined at least 4 times. 
Additional projects in the full population of 778 mayor IT projects may also have been 
rebaselined at least 4 times. 
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Department 

Project 

Number of times 
rebaselined 

U.S. Department of 
Agriculture 

Program Fund Control 

System 

5 

Department of Commerce 

Patent and Trade Office 

Revenue and Account 
Management System 

5 

Department of Commerce 

Commerce Business 

Environment 

5 

Department of Veterans 
Affairs 

Health Admin Center IT 
Operations 

6 

Department of Housing and 
Urban Development 

Tenant Rental Assistance 
Certification System 

7 


Source: GAO analysis of agency survey responses 


Agency officials reported that the key reasons for the most recent 
rebaselinings were changes in project requirements, objectives, or 
scope, and changes in funding stream. Table 6 shows the estimated 
frequencies of each of these reasons. 


Table 6: Estimated Frequency of Reasons for the Most Recent Rebaselining of 
Projects 


Category of Reasons 

Percentage of Times 
Reported 

Change in project requirements, objectives, or 
scope 

55% 

Chanqe in fundinq stream 

44% 

Original baseline was inaccurate 

14% 

Cost or schedule overruns due to project 
performance 

4% 

Cost or schedule overruns due to contractor 
performance 

4% 

Other 

41% 


Source: GAO analysis of agency survey responses 

Note: Percentages do not total 100 percent because multiple reasons could be provided for 
rebaselining projects. 


Several rebaselined projects we have performed detailed reviews of 
have experienced significant cost or schedule changes. For 
example, the U.S. Coast Guard’s Rescue 21 program is projected to 
have cost increases of 184 percent and schedule delays of 5 years 
after rebaselining. Table 7 provides additional examples of projects 
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we have reviewed that experienced significant cost or schedule 
changes. 


Table 7: Rebaselined Projects’ Cost and Schedule Changes (in millions) 


Project 

Original 

Cost 

Cost After 
Rebaseline(s) 

Dollar 

Change 

Percent 

Change 

Original 

Completion 

Date 

Completion 
Date After 
Rebaseline 

Delay 

National Polar- 
orbiting 

Operational 
Environmental 
Satellite System" 

$7000 

$ 12500 

$5500 

79% 

2018 

2026 

8 years 

Navy Enterprise 
Resource 

Planning 

$ 1993 

$2445 

$452 

23% 

Jun. 2011 

Aug. 2013 

2.2 years 

FAA Standard 
Terminal 

Automation 

Replacement 

System 

$940 

$2770 

$1830 

195% 

Oct. 2005 

Dec. 2007 

2.2 years 

FAA Wide Area 

Augmentation 

System 

$1001 

$3340 

$2339 

234% 

Aug. 1999 

Dec. 2008 

9.3 years 

US Coast Guard 
Rescue 21 

$250 

$711 

$461 

184% 

2006 

2011 

5 years 


Source: GAO reports and agency data 


“Only a portion of this program's costs are included in the $70 billion estimated cost. The rest is not 
considered to be an TT investment. 


Agencies’ Rebaselining Policies Are Not Comprehensive 

We are also reporting that, although the 24 m^jor agencies have 
rebaselined about half of their major IT projects that they planned to 
invest in during fiscal year 2008, they have not been guided by 
comprehensive rebaselining policies. Specifically, while m^jor 
agencies have all established rebaselining policies, none of the 
policies are fully consistent with best practices such as describing a 
process for developing anew baseline. 


Page 28 


GAO-08-1015T 





80 


Our recently issued draft GAO Cost Assessment Guide * includes five 
practices that are relevant to rebaselining policies: 

1. Describe reasons when a rebaseline is warranted. A rebaselining 
policy should require valid reasons for rebaselining such as that 
the baseline is no longer useful as a management tool (e.g., 
cost/schedule variances are so high that they lose meaning; 
program scope has significantly changed). 

2. Describe the process for developing anew baseline. A 
rebaselining policy should describe the development of a new 
cost estimate and a new project plan that details the scope of the 
remaining work along with schedule and resource allocation. 

3. Require validating the new baseline. A rebaselining policy should 
identify who can validate the new baseline and how the 
validation is to be done. 

4. Require management review. A rebaselining policy should 
identify the authority who decides whether the rebaselining is 
warranted and the rebaselining plan is acceptable. In addition, 
the policy should outline decision criteria used by the decision 
authority to determine if the rebaseline plan is acceptable. 

5. Require that the process is documented. A rebaselining policy 
should identify and document rebaselining decisions, including 
the reasons for rebaselining; changes to the approved baseline 
cost, schedule, and scope; management review of the rebaseline 
request; and approval of new baseline. The policy should also 
require an explanation of why the current plan is no longer 
feasible; identify the problems that led to the need for a new plan 
of the remaining work; and discuss measures in place to prevent 
recurrence. 

Our analysis shows that agencies do not have comprehensive 
rebaselining policies. Specifically, none of the agencies' rebaselining 
policies are fully consistent with all of the five practices mentioned 
above. Most policies fully or partially addressed reasons for 
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rebaselining, requiring management review, and requiring that the 
rebaselining process be documented (79 percent, 96 percent, and 88 
percent, respectively), while describing the process for developing 
the new baseline, and requiring validation of the new baseline were 
addressed the least (46 percent and 54 percent of the policies, 
respectively did not address these practices). Table 4 summarizes 
our assessment of agencies’ rebaselining polices and table 5 
provides a detailed assessment by agency. 


Table 4: Summary of Rebaselining Policy Assessment 


Extent to Which Policy Addressed Best Practices 


Practice 

Number (and 
percent) of 
Policies that 
Fully 
Addressed 
the Practice 

Number (and 
percent)of 
Policies that 
Partially 
Addressed 
the Practice 

Number(and 
percent) of 
Policies that 
Did Not 
Address the 
Practice 

Describe reasons when a rebaseiine 
is warranted 

14(58%) 

5(21%) 

5(21%) 

Describe process for developing a 
new baseline 

0 (0%) 

13(54%) 

11 (46%) 

Require validating the new baseline 

5(21%) 

6(25%) 

13(54%) 

Require management review 

9 (38%) 

14 (58%) 

1 (4%) 

Require that the process is 
documented 

6 (25%) 

15(63%) 

3(13%) 


Soocrce: GAO analysis of agencies' rebasbnmg policies 


Agencies’ policies vary in part because no guidance has been issued 
specifying what elements these policies are to include. As previously 
noted, OMB has issued guidance which, among other things, 
requires baseline change requests to be approved by the agency 
heads and to be submitted to OMB for approval. However, this 
guidance does not specifically address how agencies are to 
implement their rebaselining activities, including the key elements 
that should be addressed in their policies, hi addition, officials from 
OMB’s Office of E-govemment and Information Technology and the 
Acting Chief of OMB’s Information Policy and Technology Branch 
told us that, in their oversight function, they review agencies’ earned 
value management policies, and in doing so determine whether 
these policies address rebaselining. However, they noted that they 
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have not established specific criteria to evaluate the EVM policies 
(and therefore their rebaselining aspects) and acknowledged that 
having such criteria would improve consistency among the policies 
and facilitate their oversight process. Without comprehensive 
policies to guide their rebaselining activities, agencies may not be 
optimizing the effectiveness of rebaselining as a tool to improve 
performance management. In addition, their rebaselining process 
may lack the transparency needed to ensure effective oversight. 

To address the weaknesses identified with agencies’ rebaselining 
policies, we made recommendations to the Director of OMB and to 
the 24 major agencies. Specifically, we recommended that 

• the Director of OMB issue guidance for rebaselining policies that 
would include a minimum set of key elements, taking into 
consideration the criteria used in our report, and 

• each of the heads of the 24 major agencies direct the 
development of comprehensive rebaselining policies that 
address the weaknesses we identified. 

We received comments on a draft of our report from 20 of the major 
agencies-3 of which stated that they had no comments. Of the 
remaining 17 agencies, 5 agreed with our recommendations, and 6 
disagreed with our assessment of their rebaselining policies and 
provided information which we have incorporated into the report, 
as appropriate. Six agencies provided varied comments, including 3 
who stated they plan to use the practices identified in our report to 
improve their policies. 


In summary, effective management and oversight of federal IT 
projects remains a crucial task for OMB and executive branch 
agencies. Hundreds of projects, amounting to billions of dollars in 
expenditures, have been identified as poorly managed, poorly 
performing, or both. While OMB has taken steps to improve the 
identification of poorly managed and poorly performing projects, 
more needs to be done to improve management and oversight, as 
evidenced by the number of recurring Management Watch List 
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projects and the surge of these projects at the beginning of every 
fiscal year. 

In addition, without sound policies guiding agencies’ rebaselining 
efforts, changes to projects’ cost and schedule goals are not as 
transparent as desired and may in fact mask cost overruns and 
schedule delays. Having sound rebaselining guidance from OMB and 
more diligent oversight of rebaselining efforts from federal agencies 
will result in more accurate information on cost and schedule 
performance shortfalls and provide the necessary transparency to 
agency officials, OMB, and other oversight organizations. As we 
transition to a new administration, it is essential to maintain the 
current momentum of identifying troubled projects and the reasons 
they are poorly planned and/or performing and to continue to focus 
attention more keenly on solutions and long-term improvement 
efforts. 

Mr. Chairman, this concludes my statement. I would be happy to 
answer any questions at this time. 


GAO Contact and Acknowledgements 

If you should have any questions about this testimony, please 
contact me at (202) 512-9286 or by e-mail at povmerd@gao.gov. 
Individuals who made key contributions to this testimony are Sabine 
Paul, Assistant Director; Neil Doherty; Lee McCracken; Kevin Walsh 
and Eric Winter. 
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Appendix I: Management Watch List Projects 

The following provides additional detail on the investments 
comprising OMB’s Management Watch List as of July 2008. The 
project names were taken from OMB’s Management Watch List 
released earlier this month and matched to the list of projects in the 
OMB’s Report on IT Spending for Fiscal Years 2007, 2008, and2009 
to derive the requested amounts for fiscal year 2009. 


Table 4: Management Watch List Projects by Agency 

Agency 

Project Name 

Fiscal year 2009 
request (in millions) 

Department of 
Agriculture 

Conservation Program Delivery 

11.11 


Multi-Family Management 

5.355 


Program Funds Control System 

6.267 


Store Tracking and Redemption System 

5.24 


Natural Resource Information, Inventory, & Assessment 

19.39 


Guaranteed 

16.674 


Integrated Program Accounting System 

3.24 


USDA Forest Service Automated Timber Sales Accounting 

1.289 


Water and Climate Information System 

2.392 


Corporate Financial Management Systems 

59.559 


Legacy Payroll/Personnel System 

10.601 


Consolidated General Sales Manager 

2.378 


Consolidated Farm Loan Program Information and Delivery System #103 

12.541 


Natural Resource Manager 

33.909 


Corporate Property Automated Information System 

2.153 


USDA Identity And Access Management 

20.757 


Consolidated Financial Management Information Systems 

7.94 


Processed Commodities Inventory Management System 

8.27 


Commerical 

7.887 


Forest Service National Financial Applications 

5.363 


Resource Ordering and Status System 

11.126 


Fire Program Analysis System - Phase 2 

2.202 


Human Resources Line of Business: Service Center 

53.851 


Integrated Acquisition System 

17.485 


Wide Consolidated Infrastructure, Office Automation, and Telecommunications 

871.714 


National Animal Identification System 

4.627 


Public Health Information Consolidation Projects 

4.65 


RMA-01 Financial Management Systems 

1.914 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions) 


Consumer 

13.489 


NASS Survey Processing System 

4.202 


Corporate insurance information System 

6.487 


APHIS Comprehensive Electronic Permit System 

3.132 


Geographic Information System 0084 

14.777 


Farm Program Modernization 

134.155 


Web Based Supply Chain Management 

21.646 


RMA-13 Emerging Information Technology Architecture - Electronic Written Agreement 
Project 

8.757 


Financial Management Modernization initiative 

46.237 

Department of 
Commerce 

Economic Census and Surveys 

61.941 


American Community Survey 

34.07 


BE A Estimation Information Technology System 

10.156 


NIST-wide Grant Management Information System 

1.25 


Demographic Surveys Statistical IT Support 

14.601 


Geographic Support Systems 

24.089 


ITA International Trade Process Streamlining 

0.76 


NPOESS Data Exploitation 

2.455 


Satellite Operations Control Center Command and Data Acquisition 

37.938 


Field Support Systems 

40.618 


The Longitudinal Employer-Household Dynamics Program 

5.69 


Advanced Weather Interactive Processing System 

57.129 


Next Generation Weather Radar System Product Improvement 

8.376 


Next Generation Weather Radar Operations and Maintenance 

8.654 


COOP Historical Climate Network - Modernization 

3.734 


NWS Dissemination Systems 

5.838 


NOAA National Data Centers 

77.71 


GOES Ground System 

19.744 


Office of Satellite Data Processing and Distribution Systems Critical Infrastructure 
Protection 

2.772 


Environmental Satellite Processing Center 

26.657 


USPTO Consolidated Financial System 

24.987 


Commerce Business Environment 

3.97 


MAF/TIGER Enhancements 

18.344 


Search and Rescue Satellite-Aided Tracking 

3.605 


Nautical Charting System 

3.879 


PORTS & NWLON 

5.367 


NPOESS Ground System 

47.115 


GOES-R Series Ground Segment 

68.939 


Data Access and Dissemination System 

48.767 


BIS Legacy Export Control 

2.475 


NOAA Non-Core CBS Financial Management System 

0.999 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions) 


National Weather Service Telecommunication Gateway System (Legacy, Replacement, 
and CIP) 

21.058 


NOAA Weather Radio All Hazards Weather Network) aka All Hazards Emergency 
Message Collection System 

0.75 


National Air Quality Forecast Capability 

6.46 


NWS Office of Hydrologic Development 

4.501 


NWS Regions & Field 

21.9 


National Integrated Drought Information System Implementation 

3.8 


NCEP Weather and Climate Operational Supercomputer Systems (WCOSS Primary and 
Backup) 

22.369 


NCEP Weather and Climate Computing Infrastructure Services 

30.708 


NOAA R&D High Performance Computing System 

26.524 


NOAA Grants On-line 

1.496 


USPTO Revenue Accounting and Management System 

7.361 


USPTO Patent File Wrapper Program 

17.887 


Commerce Business Systems (formerly Commerce Administrative Management System) 

39.313 


Department of Commerce Consolidated IT Infrastructure 

432.269 


EDA Operations Planning and Control System and Loan Billing and Management System 

0.66 


BIS ECASS2000+ 

5.316 


OCIO Financial Management IT Operations 

8.13 


NWS/ Weather Radio Improvement Project 

5.74 


Comprehensive Large Array-data Stewardship System 

8.966 


NOAA Research Scientific Computing Support 

19.204 


Geodetic Support System 

1.815 


POES Ground System 

15.274 


USPTO Business Continuity and Disaster Recovery Program 

6.421 


NIST Central IT Support for Science 

8.737 


Radio Spectrum Management: Federal Spectrum Management System 

3.074 


NDBC Ocean Observing System of Systems 

7.125 


Decennial 2010 Systems Design and Integration, and Decennial 2010 Testing and 
Evaluation 

604.159 


BIS ECASS Modernization 

0.357 


USPTO First Action System for Trademarks 

1.891 


Automated Surface Observing System Operations and Maintenance 

2.95 

Department of 
Defense 

Logistics Modernization Program 

199.142 


DeCA Enterprise Business System 

30.293 


Integrated Data Environment/Global Transportation Network Convergence 

38.018 


Global Combat Support System - Marine Corps 

52.488 


Warfighter Information Network-Tactical 

616.028 


Global Combat Support System - Army 

139.149 


Defense Enrollment Eligibility Reporting System 

146.218 


General Fund Enterprise Business System 

130.458 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions) 


Advanced Field Artillery Tactical Data System 

51.988 


FORCE XXI Battle Command Brigade And Below 

297.234 


GUARDNET XXI, The Army National Guard's Wide Area Network 

74.059 


Mounted Battle Command on the Move Program 

37.697 


Transportation Coordinators’ Automated Information For Movements System II 

69.879 


Expeditionary Combat Support System 

275.227 


Mission Planning Systems 

214.151 


Integrated Strategic Planning And Analysis Network 

67.192 


Minimum Essential Emergency Communications Network 

82.47 


Battle Control System - Mobile 

194.972 


Global Combat Support System - Air Force 

76.088 


Defense Medical Human Resource System internet 

27.061 


Defense Integrated Military Human Resources System 

63.37 


Standard Procurement System 

22.225 


Global Decision Support System 

42.926 


DOD TELEPORT 

32.666 


Net-Enabled Command Capability 

0 


Protect Information - Public Key Infrastructure 

42.329 


Navy Enterprise Resource Planning 

177.116 


Common Aviation Command And Control System 

76.722 


Navy Marine Corps Intranet 

1609.615 


US Mepcom Integrated Resource System 

65.51 


Installation Information Infrastructure Modernization Program 

233.303 


Global Command And Control System - Army 

34.842 


Combat Information Transport System 

341.706 


Battle Control System Fixed 

65.911 


Cheyenne Mountain Complex/Tactical Warning - Attack Assessment 

90.977 


Commissary Advanced Resale Transaction System 

15.49 


Military Computer-Based Patient Record (includes Inits 0049, 0379, and 0435) 

0 


Executive Information/Decision Support 

66.37 


Defense Message System 

67.896 


Defense Information System Network 

1518.426 


Global Combat Support System-COCOM-JTF 

39.224 


Joint Precision Approach And Landing System 

99.929 


Global Command And Control System - Maritime 

85.214 


JTRS - Airborne, Maritime And Fixed Radios 

204.454 


Deployable Joint Command And Control 

34.632 


Future Combat Systems-Advanced Collaborative Environment 

22.565 


Distributed Learning System 

62.19 


Maneuver Control System 

161.517 


Defense Enterprise Accounting and Management System-Air Force 

59.968 


Air and Space Operations Center - Weapon System 

288.144 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions) 


Theater Battle Management Core System 

70.27 


Defense Medical Logistics Standard Support 

56.531 


Global Command And Control System- Joint 

137.458 


Theater Medical Information Program-Joint 

66.287 


Net Centric Enterprise Services 

127.091 


JTRS - Handheld, Manpack, And Small Form Fit Radios 

164.766 


JTRS - Network Enterprise Domain 

244.165 


Defense Enterprise Accounting And Management System 

11.746 


Future Business System 

32.067 


Defense Travel System 

27.417 


Key Management Infrastructure 

49.578 


JTRS - Ground Mobile Radios 

196.32 


Defense Information System For Security 

31.9 

Department of 
Education 

Federal Student Aid Financial Management System 

10.187 


E-Authentication 

3.163 


Common Services for Borrowers-Legacy 

184.714 


Information Assurance 

8.637 


Enterprise Portal 

5.155 


National Student Loan Data System 

9.85 


ED Web 

5.686 


Common Origination and Disbursement 

45.307 


Virtual Data Center 

24.308 


ADvance - Aid Delivery 

69.326 


ADvance - Person Data Management Program 

23.01 

Department of 
Health and 
Human Sen/ices 

HHS Consolidated Acquisition Solution 

9.534 


IHS Resource and Patient Management System - Maintenance & Enhancements 

73.678 


National Select Agent Registry (formerly SATERIS) 

7.488 


Electronic Research Administration 

39.699 


CMS Integrated Data Repository 

7.485 


Consolidated Infrastructure 

111.551 


Medicaid Data Systems 

7.552 


Data Management Operations - Claims 

15.737 


OS ASAM IT Service Center 

57.594 


Information and Computer Technologies for the 21st Century 

19.233 


CMS Drug Claims 

22.333 


CMS if Infrastructure 

259.084 


ACF GrantSrtutions.gov / Grants Administration Tracking Evaluation System - Grants 
Center for Excellence 

4.964 


OS OPHS Commissioned Corps Force Management Solution 

4 


HHS Asset - Property Management Information System 

1.337 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions) 


HHS HR LOB IT 

0 


information Technology Infrastructure 

76.1 


interoperability & Standardization - Provider ID 

4.27 


Federal Health Architecture 

3.662 


NIH Business System 

25.06 


CMS Modernized IT Infrastructure 

11.25 


CMS ICD-10 initiative 

17.15 


OS ASAM Payment Management System 

7.03 


OS ASAM Debt Management and Collection System 

5.728 


OS ONC Prototype Nationwide Health Information Network Architectures 

19.127 


NIH IT Infrastructure 

282.496 


AHRQ Medical Expenditures Panel Survey 

21.587 


OS ASAM Accounting for Pay System 

2.267 


IHS Infrastructure, Office Automation, & Telecommunications 

19.146 

Department of 

Homeland 

Security 

ICE - Federal Financial Management System 

30.379 


FAMS Air to Ground Communications System and Tactical Information Sharing System 

12.8 


NPPD - Information Systems Security Line of Business 

2.577 


USCIS - Immigration - CLAIMS 3.0 

11.176 


CBP - Advance Passenger Information System 

7.94 





USCIS - Transformation 

71 


USCIS - Naturalization - CLAIMS 4.0 

18.271 


Secure Border Initiative net: Command, Control, Communications & Intelligent Systems 

157 


Crew Vetting 

20.925 


Vessel Logistics System 

3.93 


Marine Information for Safety and Law Enforcement 

13.144 


Integrated Common Analytical Viewer 

4.864 


FEMA - Disaster Management E-Govemment Initiative 

12.714 


DHS - Homeland Secure Data Network 

47.673 


DHS - Infrastructure 

1461.074 


Rescue 21 

126.65 


Integrated Deepwater System 

14.4 


FEMA - eNEMIS 

12.688 


CBP - Traveler Enforcement Compliance System - Modernization 

65.7 


ICE - Detention and Removal Modernization 

13.372 


Integrated Deepwater Systems 

92.1 


NPPD - IICP - Infrastructure Information Collection Program 

14.643 


DHS - Financial Transformation & Systems Consolidation 

19.2 


USCIS - integrated Document Production 

31.855 


NBIS National Bio-Surveillance Integration System 

3.125 


NPPD - US-CERT 

109.154 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions) 


CBP - Non-lntrusive Inspection Systems Program 

159.371 

Department of 
Interior 

USGS - landsat Data Continuity Mission 

24.15 


OS - OHTA Account Reconciliation Tool 

3.135 


DOI - Advanced Budget/Accounting Control and Information System 

0.685 


USGS - Landsat 

16 


BIA - Integrated Records Management System 

2.67 


DOI - Consolidated Financial Statement System 

0.765 


Minerals Revenue Management Support System 

21.278 


USGS - National Water Information System 

7.46 


BIA - LOMAS - Loan Management and Accounting System 

0.3 


E-DOI - Geospatial One-Stop 

1.65 


DO! - Capstone Facility Management System 

9.625 


OSM - Abandoned Mine Land Inventory System 

0.175 


USGS - The National Map Reengineering Project 

4.829 


National Fee Collection Point of Sale System 

0.93 


BIE - Native American Student Information System 

2.468 


DOI - Federal Financial System 

26.448 


DOI - Financial and Business Management System 

92.705 


E-DOI - NBC FMLoB Shared Service Provider 

0 


E-DOl - NBC Shared Service Center 

21.001 


BLM-National Integrated Land System 

2.943 


BLM-Incident Qualifications and Certification System 

1.337 


OSM - Applicant Violator System 

0.256 


OSM - Coal Fee Collection Management System 

0.404 


BOR1-PABS (Program and Budget System) 

1.042 


BOR1-CDW (Corporate Data Warehouse) 

0.658 


BOR1-RMSS (Reclamation Mission Support System) 

42.158 



2.338 


BIA - Trust Asset Accounting Management System 

6.076 


BIA - Facilities Management Information System 

1.5 


BLM-Collections and Billings System 

2.06 


DOI - Interior Department Electronic Acquisition System 

5.019 


BLM-Legacy Rehost 

1.84 


BLM-Automated Fluid Mineral Support System 

1.599 


BLM-IT Support for Resources and Mineral Land Use Planning 

0.55 


MMS - OCS Connect 

5.647 


USGS * National Biological Information Infrastructure 

4,207 


USGS - Advanced National Seismic System 

8 


NPS -NPS.gov Internet/Intranet Portal: Infrastructure - Public Web Services 

2.86 


E-DOI - Geospatial Line of Business 

0.372 


National Fire Plan Operations and Reporting System 

0.525 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions) 


FWS - Federal Aid Information Management System 

2.409 


E-DOi - Recreation 

0.2 


BIA - National Irrigation Information Management System 

2.195 


OST - Trust Funds Accounting System 

14.7 


DOI - Incident Management, Analysis, and Reporting System 

11.779 


DOI - Consolidated infrastructure, Automation, Telecomm 

495.843 


BOR1-CVACS (Central Valley Automated Control System) 

1.371 


BOR1-HSCADA (Hoover Supervisory Control and Data Acquisition System 

0.826 


BOR1-GCPO SCADA (Grand Coulee Power Office Supervisory Control and Data 
Acquisition System) 

4.065 


BOR1-CRSP SCADA (Colorado River Storage Project Supervisory Control and Data 
Acquisition System) 

0.623 

Department of 
State 

Joint Financial Management System 

18.261 

Department of 
Transportation 

DOTXX070: DOT IT Combined Infrastructure 

232.14 


FAAXX712 - Next Generation Air Transportation System 

649.185 

Department of 
Treasury 

Enterprise IT Infrastructure Optimization Initiative 

1466.792 


Treasury-Wide Enterprise Content Management Services 

28.168 


Consolidated Enterprise Identity Management Project 

59.469 


Fiscal Management 09 

1.92 

Department of 
Veterans Affairs 

Decision Support System 

19.217 


VistA Imaging 

29.431 


VA-Learning Management System 

6.379 


VistA-Application Development 

146.301 


One-VA Registration and Eligibility 

7.296 


Health Data Repository 

28.916 


VistA-Foundations Modernization 

115.366 


Virtual VA 

18.12 


Benefits Processing and Workflow 

2.272 


Enrollment Enhancements 

19.687 


Federal Health Information Exchange 

6.53 


My HealtheVet 

20.467 


USA Staffing 

5.129 


VistA-Legacy 

360.414 


Pharmacy Re-Engineering and IT Support 

19.5 


Enrollment Operations and Maintenance 

0.853 


Veterans Benefits Delivery 

2.169 


Benefits Delivery Network Maintenance and Operations 

22.426 


e-Payroll 

8.039 


VBA Application Migration Program 

0 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions} 


VA-Wide e-Travel Solution 

1.365 


Scheduling Replacement Project 

32.609 


VistA Laboratory IS System Reengineering 

30.925 


VBA Rules Based Claims Processing 

6.288 


Capital Asset Management System 

2.801 


One VA Contact Management 

7.658 


Health Admin Center IT Operations 

21.815 


VETSNET 

30.424 


Benefits Support Services 

42.719 


Program Integrity/Data Management 

12.512 


Revenue Improvements and System Enhancements 

5.4 


Persona! Identification Verification 

17.887 


Medical and Prosthetic Research Operations 

24.77 


Document and Correspondence Management System 

1.431 


Financial & Logistics Integrated Technology Enterprise 

45.05 


Financial Management System 

15.506 


IT Infrastructure 

1000.813 


Allocation Resource Center 

1.392 


Biood Bank 

1.961 


Payroil/HR Systems 

41.428 

Corps of 
Engineers 

ENGLink Interactive 

2.81 


Automated Personal Property Management Systems 

0.51 


Real Estate Management Information System 

3.48 


Corps of Engineers Financial Management System 

9.59 


Operations and Maintenance Business information Link 

2.5 


Resident Management System 

2.554 


Project Management Information System II 

14.3 


Corps Water Management System 

1.936 


Consolidated Information Technology Infrastructure/Office 
Automation/Telecommunications 

304.367 


Facilities & Equipment Maintenance System 

3.5 

Office of 
Personnel 

Management 

Retirement Systems Modernization 

39.94 

Nuclear 

Regulatory 

Commission 

National Source Tracking System 

7.902 


Licensing Support Network 

2.501 


Budget Formulation System 

0.537 


Secure LAN and Electronic Safe 

4.098 


Reactor Program System 

1.294 


Incident Response System 

3.97 
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Agency 

Project Name 

Fiscal year 2009 
request (in millions) 


Enterprise Digital Data Management System 

0.081 


Cost Accounting System 

0.733 


Core Financial System - Replacement 

5.701 


License Fee Billing System 

1.904 


Material Licensing Program - Web-Based Licensing 

1.285 


Infrastructure Services and Support 

67.476 


Time and Labor Legacy 

0 


Time and Labor Modernization 

1.646 


Agency-wide Documents Access and Management System 

8.707 


Source: OMB Management Watch List released in July 2008 for project names and OMB’s 
Report on IT Spending for Fiscal Years 2007, 2008, and 2009 for the financial data 
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Appendix II: Reasons for Inclusion on the Management Watch List 
by Agency 

The following table provides additional detail on the frequency of 
the reasons for inclusion on the Management Watch List for each 
agency for the projects remaining on the List as of July 2008. It 
shows security and cost and schedule performance as being the 
most common reasons. 


Table: Frequency of Reasons for Inclusion on the Management Watch List by Agency (as of July 2008) 


Agency 

Presi- 

dent's 

Manage- 

ment 

Agenda 

Program 

Manage- 

ment 

Acquisi- 

tion 

Strategy 

Perfor- 

mance 

Infor- 

mation 

Security 

Privacy 

Enter- 

prise 

Arcitec- 

ture 

Alter- 

native 

Analysis 

Risk 

Manage- 

ment 

Cost and 
Schedule 
Perfor- 
mance 

Department of 
Agriculture 

0 

0 

0 

0 

0 

2 

3 

0 

0 

2 

Department of 
Commerce 

0 

0 

0 

0 

1 

0 

0 

0 

0 

5 

Corps of 
Engineers 

0 

1 

1 

1 

0 

0 

1 

1 

1 

2 

Department of 

Homeland 

Security 

0 

9 

9 

5 

7 

4 

2 

8 

0 

9 

Department of 
Defense 

1 

2 

1 

1 

1 

2 

1 

1 

3 

9 

Department of 
Transportation 

0 

1 

0 

1 

1 

1 

0 

1 

1 

1 

Department of 
Education 

0 

3 

4 

1 

7 

3 

1 

6 

1 

4 

Department of 
Health and 

Human Services 

5 

1 

12 

1 

10 

0 

0 

4 

5 

9 

Nuclear 

Regulatory 

Commission 

0 

1 

0 

0 

13 

1 

0 

0 

1 

0 

Office of 

Personnel 

Management 

0 

0 

1 

0 

1 

0 

0 

1 

1 

1 

Department of 
State 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

Department of 
Treasury 

0 

0 

1 

0 

3 

4 

0 

1 

1 

2 
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Agency 

Presi- 

dent’s 

Manage- 

ment 

Agenda 

Program 

Manage- 

ment 

Acquisi- 

tion 

Strategy 

Perfor- 

mance 

infor- 

mation 

Security 

Privacy 

Enter- 

prise 

Arcitec- 

ture 

Alter- 

native 

Analysis 

Risk 

Manage- 

ment 

Cost and 
Schedule 
Perfor- 
mance 

Department of 
Veterans Affairs 

1 

2 

22 

2 

26 

36 

0 

6 

9 

40 

Total 

7 

20 

51 

12 

71 

54 

8 

29 

23 

84 


Source: GAO analysis of OMB’s July 2008 Management Watch list data. 
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Appendix III: High Risk Projects with Shortfalls 

The following provides additional detail on the high risk projects 
that have performance shortfalls as of June 2008, These shortfalls 
were identified by agencies in June 2008 high risk reports. (Note: the 
Department of Veterans’ Affairs did not provide its report.) The 
project names were taken from OMB’s list of Management Watch 
List released earlier this month and matched to the list of projects 
on agencies’ June 2008 high risk reports. 


Table 5: High Risk Projects with Shortfalls by Agency 

Agency Investment Name 

Fiscal Year 
2009 

request (in 
millions) 

Unclear 

baselines 

Cost and 
schedule 
variance 
exceeding 
10% 

Project 

manager 

not 

qualified 

Duplication 

Department of NPPD HCP 

Homeland Security 

15 

X 

X 

X 


NPPD- US Cert 

109 

X 


X 

X 

DHS Transformation & Systems 

Consolidation 

19 



X 


S&T Disaster Management E-Gov 

13 


X 

X 


DHS - HR IT 

17 

X 




DHS Infrastructure 

1,461 

X 

X 



DHS - Homeland Secure Data Network 

48 


X 

X 


A&O Homeland Security Information Network 

21 

X 

X 



USCIS- Transformation 

71 

X 


X 


USCIS Immigration - CLAIMS 3.0 

11 


X 



ICE Federal Financial Management System 

30 


X 

X 


CBP Secure Border Initiative net Technology 
Program 

157 

X 

X 



Secure Flight 

83 


X 



TSA Transportation Worker Identification 
Credentialing 

27 


X 



TSA Crew Vetting 

21 


X 

X 


TSA- FAMS Air to Ground Communications 
& Tactical Information Sharing 

13 


X 

X 


ICE Automation and Modernization 

23 

X 

X 



USCG-Core Accounting System 

13 


X 



USCG- Vessel Logistics System 

4 



X 


USCG- Marine Information for Safety and law 
Enforcement 

13 


X 

X 
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Agency 

Investment Name 

Fiscal Year 
2009 

request (in 
millions) 

Unclear 

baselines 

Cost and 
schedule 
variance 
exceeding 
10% 

Project 

manager 

not 

qualified 

Duplication 


USCG Nationwide Automatic identification 
System for MDA 

38 


X 




NPPD US-VISIT 

453 


X 




NPPD Information Systems Security Line of 
Business 

3 

X 

X 

X 

X 


FEMA -Integrated Financial Management 
Information System 

3 




X 


FEMA eNEMIS 

13 


X 

X 


Department of 
Commerce 

FDCA 

221 

X 

X 



Department of 
Defense 

E-Training 

NA 

X 

X 


X 


Recruitment One Stop - Migration 

NA 

X 

X 


X 


EHRI - Migration 

NA 

X 

X 


X 


E-Training - Legacy System 

NA 

X 

X 


X 

Department of 
Energy 

EE Corporate Management and Planning 
System 

2 


X 



Department of 
Interior 

DOI - IMARS 

12 

_ 

X 




BLM - ePIanning 

1 


X 




MMS - OCS Connect 

6 


X 



Department of 
Justice 

Unified Financial Management System 

123 


X 




SENTINEL 

97 


X 




Case Management LOB 

NA 

X 

X 

X 

X 


E-Travel Migration 

NA 


X 




E-Authentication Implementation 

NA 

X 

X 

X 

X 


EHRI (eOPF conversion) 

NA 

X 

X 

X 

X 


HR LoB - Legacy System 

0 

X 

X 

X 

X 

Department of 
Labor 

EFAST2 

7 


X 



Department of 
Transportation 

E-Authentication Migration 

NA 


X 




FAAXX705: Traffic flow Management 

114 


X 




FAAXX445: FAA Telecommunications 
Infrastructure 

188 


X 



Department of 
Education 

ADvance Person Data Management 

NA 


X 




Grants Administration Payment System 

4 



X 



G5 

7 


X 

X 



Common Origination and Disbursement 

45 



X 
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Agency 

Investment Name 

Fiscal Year 
2009 

request (in 
millions) 

Unclear 

baselines 

Cost and 
schedule 
variance 
exceeding 
10% 

Project 

manager 

not 

qualified Duplication 


Integrated Technical Architecture/ Enterprise 
Application Integration 

8 



X 


Enterprise Information System 

0 


X 

X 


integrated Partner Management 

to 


X 



ADvance (Aid Delivery) 

69 


X 

X 

Environmental 
Protection Agency 

E-Travel Migration 

1 


X 



FM LOB - Migration 

31 


X 


General Service 
Administration 

HRLOB Migration 

NA 

X 



Department of 
Health and Human 
Services 

Federal Health Architecture 

4 


X 



FDA MedWatch Plus 

12 


X 



OS OPHS Commissioned Corps Force 
Management Solution 

4 


X 


Office of Personnel 
Management 

RSM 

40 

X 



Department of 
Treasury 

Correspondence Examination Automation 
Support Major Wintel 

8 


X 



Correspondence Imaging System - Major 

7 


X 



Counsel Automated Systems Environment 

28 



X 


Enterprise Data Access Strategy 

6 


X 



Examination Desktop Support System - 
Release 2 - Major 

8 


X 



Excise Files Information Retrieval Systems 

11 


X 



Financial Information and Reporting 
Standardization 

7 


X 



GAISS 

4 


X 



Government-Wide Accounting and Reporting 
Modernization 

19 


X 



HR LoB - HR Connect 

23 


X 



Integrated Financial System/CORE Financial 
System 

15 



X 


Integrated Submission and Remittance 
Processing System 

17 


X 



Oracle e-Business Suite 

27 


X 



Pay.gov 

9 


X 



Payment Application Modernization 

19 


X 



SaBRe 

5 


X 



Service Center Recognition Image 

Processing System 

16 


X 
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Agency 

Investment Name 

Fiscal Year 
2009 

request (in 
millions) 

Unclear 

baselines 

Cost and 
schedule 
variance 
exceeding 
10% 

Project 

manager 

not 

qualified 

Duplication 


Travel Reimbursement and Accounting 
System 

0 




X 


Treasury Foreign Intelligence Network 

5 


X 




TreasuryDirect 

6 


X 




Treasury-Wide Enterprise Content 
Management Services 

28 


X 



Agency for 

Internationa! 

Development 

E-Authentication 

NA 

X 

X 




E-Travel 

1 


X 




HSPD-12 

2 

X 

X 



Department of 
Agriculture 

Consolidated Infrastructure, Office 
Automation and Telecommunications 

872 

X 



X 


APHIS - National Animal Identification 
System 

5 

X 





RMA - Common Information Management 
System 

3 

X 





NA Not available: Financial data on project was not found in OMB's Report on IT 
Spending for Fiscal Years 2007, 2008, and 2009. 


Source: OMB’s Management Watch List released in July 2008 for the project names; OMB’s 
Report on JT Spending for Fiscal Years 2007, 2008, and 2009 for the financial data; and GAO 
analysis of agencies' June 2008 high risk reports for all other information. 
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Testimony of 
Alfred Grasso 

President and CEO, The MITRE Corporation 
to the 

Committee on Homeland Security and Government Affairs 
on the subject of 

"Off-line and Off-Budget: The Dismal State of Federal Information 
Technology Planning" 


Chairman Carper, Senator Coburn, Honorable Members, thank you for the opportunity 
to appear before your Subcommittee. My name is Alfred Grasso, and I am the Chief 
Executive Officer and President of The MITRE Corporation. Our company's 50 years of 
experience, contributions, and accomplishments have given us a perspective that I 
believe is highly relevant to today's topic of information technology planning and 
management. From the early days of the SAGE air defense system to present day 
deployment of advanced command and control and business modernization systems, 
MITRE has been witness to great successes and significant disappointments. We are 
honored to be asked to share our lessons and insights with your committee. 

Federal information technology (IT) programs operate in an environment of rapid 
technology evolution, where some system components become obsolete while the 
program is still in development. This pace of technology change challenges program 
teams to keep their technical skill base current. IT systems and business processes are 
increasingly interconnected within and across agencies, making it hard to achieve 
consensus on vision, operational concept, and requirements. The federal government's 
stretched fiscal and human resources further complicate the situation. The net effect is 
the widespread failure of many programs to deliver on time and on budget, with only a 
few notable exceptions where programs are able to overcome these challenges and 
succeed. Our experience leads me to comment on several critical areas and to offer 
three steps for improvement. 

My first comment pertains to governance. Governance relates to decisions that define 
expectations, grant power, assign accountability, or verify performance. Effective 
governance comprises consistent management, cohesive policies, processes, and 
decision-rights for a given area of responsibility. Governance becomes increasingly 
complicated as programs and processes cross organizational boundaries and intersect 
multiple governing bodies. Authorities and responsibilities become ambiguous and 
program managers are disenfranchised. It is often said that the debate begins in 
government once the decision is made. 

Successful programs must have unambiguous governance. Decision-making authority 
and accountability that address the implications of intersecting organizations must be 
clearly defined at the onset. These authorities must encompass the areas of budget and 
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finance, investment portfolio management, business process, program and project 
management. 

My second comment pertains to requirements, an equally important consideration. 
Requirements reality and flux are often recognized as the root cause of program re- 
baselining. Re-baselining is not a dirty word but a necessary part of delivering 
capabilities that meet the user's needs. Requirements are too often determined in the 
absence of cost, schedule, and technology risk considerations, and once determined 
they are very difficult to change. 

The biggest difference between successful commercial IT developments and troubled 
government IT acquisitions is how requirements are managed. Successful commercial IT 
developers handle requirements with great caution. If a certain requirement adversely 
drives cost, performance, or schedule it is quickly modified or eliminated. This does not 
happen in the typical government IT acquisition. Time-to-market is a competitive driver 
in the commercial marketplace, and I would submit it is as important, if not more so, in 
a world where adversary capabilities change as quickly as the technology cycle. System 
requirements must be considered "living" but managed with a controlled process using 
regular trade-off analyses to determine the value of change. 

One concept, put forth by George Spix of Microsoft drives this point home. Spix suggests 
that if one were to have a competition between a program conducted the traditional 
way (tight control over requirements process) and a program with the same objectives, 
but where the developer has full control over requirements and is provided only one 
tenth of the funding, the non-traditional program would produce a better product in a 
shorter timeframe. The many prototypes and "proof of concept" developments that 
transition directly to operation, as well as the experience of many commercial 
developments, seem to substantiate this theory. 

Another key element to rapid fielding of capability is the notion of a pipeline such as 
depicted in Figure 1. This pipeline consists of concurrent processes for Capability 
Planning; incremental Development, Integration and Test; and Architecture and 
Standards. 

• Capability Planning is the process by which products and capabilities are defined 
and time phased. This is done in close collaboration with users, partners, and 
developers. 

• Development Integration and Test is done in small, time definite, increments. 
Each development is oriented toward a specific business/mission function (or 
small set thereof). 

• Architecture and Standards enable individual development efforts to build on 
each other, eventually providing the full functionality required. A layered 
architecture, or a separation-of-concerns approach, enables technology refresh, 
requirements flexibility, and the ability to have some developmental failures. 
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while not placing the entire project at risk. In this approach infrastructure build- 
out follows capability, not vice versa. 

Throughout this process, there is close interaction among users, developers, the test 
community, and decision makers. This is analogous to the successful approach taken by 
the Missile Defense Agency in the rapid development and deployment of the Ground- 
Based Missile Defense capability. 


Figure 1 . Creating a pipeline to deliver new capabilities 


Capability 

Planning 


Capability definition 
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•Product defies boos 
•Project plans 


PMO lead, executed 
coltaboratively with 
users, partners, 
developers 
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Test 
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synchronization, 
executed by 
developers 


Architecture & 
Standards 


O- 


•Updates as requited 


Architecture and standards development 


Common design patterns 


PMO lead, executed 
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Enterprise partners , 
developers, industry 


My final comment addresses program management practices. Successful programs are 
characterized by a strong government program management office— or PMO — capable 
of a peer relationship with the contractor(s) on systems engineering and program 
management issues. With a strong and capable PMO, the government has the capability 
to make informed decisions and manage the risk in acquisition programs. A key function 
of a strong PMO is best described by the metaphor of an Architect's relationships with 
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the user and the builder of a building. The Architect is the user's agent and is 
independent of the builder. 

The Architect works to understand the user's operational needs and translate them into 
technical requirements enabling builders to develop the needed capability. The 
Architect evaluates development feasibility and performs an independent conceptual 
design and cost estimate. These Architect functions enable the user to make informed 
cost and capability tradeoffs and prioritize requirements. The Architect is accountable to 
the user to ensure that the delivered capability meets the user's highest-priority needs 
within the constraints imposed by available technology, funding, and time. 

The Architect also supports other critical decision-making processes. For example, one of the 
most important decisions a PMO makes is selection of a prime contractor. Many studies have 
concluded that contractor past performance should be a prominent factor in the source 
selection decision. An effective Architect is also instrumental in helping the PMO structure 
the source selection to effectively incorporate past performance into the decision process. 

There are many successful programs that exhibit the characteristics described above. 
One such program is the Distributed Common Ground Station-Army (DCGS-A). This 
program has fielded leading-edge technology, flattening the intelligence architecture 
across echelons and coalition partners. DCGS-A leveraged the successful Joint 
Information Operations Center-lraq proof-of-concept effort, bringing operational 
intelligence information and alerts to field units and individual soldiers today in Iraq. 
Attachment 1 elaborates on strong program offices and the Architect function. 

I offer the following recommendations based on our experience with these issues: 

First, change the tone and tenor of oversight to focus equally on programs that have 
gone from bad to good or good to great to reveal best practices, which then can be 
applied more broadly. No program is without risk. We should all be more interested in 
those programs that have managed risks well and harvest those results for the 
betterment of a larger set of programs. 

In our experience, we have seen the impact that the tone and tenor of oversight has on 
decision-making through program and enterprise governance and program operations. 
When program decisions are made in anticipation of the reaction or desires of the Office 
of Management and Budget (OMB) or Congress, the program, the stakeholders, and the 
public are not necessarily well served. While Congress must exercise its stewardship 
responsibility and authority to hold government managers accountable for 
performance, we have observed a greater focus on programs that have gone from "bad 
to worse" than those that have gone from "bad to better" and ''better to best." 

Government managers will be more effective with exposure to a structured, thoughtful 
discussion of both why some troubled programs fail and why some improve. We believe 
that Congress has a role in modeling this approach, and that OMB is an appropriate 
facilitator of these discussions. It is our belief that a variety of forums, held in public and 
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in private, where real-world experiences are discussed and analyzed to understand why 
programs improve, would serve the overall interest of improved outcomes. This hearing 
is a good example of a first step, and the Chairman and Ranking Member should be 
commended on their approach— focusing hearings on solutions as opposed to problems 
and highlighting the struggling programs. 

Specific recommendations include, but are not limited to: 

• OMB-chaired, facilitated workshops on a variety of program delivery topics, 
attended by a cross-section of program leaders and government technical 
professionals, highlighting program cases as examples, and held in low-key, 
private venues that encourage discussion of issues and successes 

• A cross-government "PMO Council," following the concept of the CFO and CIO 
councils, constituted as a forum for Program Leaders to work together to 
establish government standards, to make recommendations directly to Congress 
on future legislative steps to help advance the state of the practice in 
government IT acquisition, and to leverage successes across the government 

The challenge of program management is to deliver programs while managing 
appropriate risk. Today this is impacted by anticipating the reaction of oversight and by 
limited forums where program-level professionals can learn from successes and see 
successes highlighted. We recommend a more enabling oversight role in the future- 
modeled by Congress, executed by OMB, and supported by the Government 
Accountability Office (GAO). We are working on additional thoughts and will be 
approaching OMB very soon to discuss these and other observations and ideas on how 
to develop and implement a program of this nature. 

Second, to navigate the dynamics and uncertainty of today's environment, IT programs 
are best structured as a portfolio with internal planning and management flexibility. 
Oversight should focus on the long-term funding envelope and the overall capabilities to 
be delivered. This allows flexibility at the program level to make informed trade-off 
decisions and to concentrate on manageably-sized increments that deliver capabilities in 
shorter time frames. This approach makes it easier for programs to demonstrate success 
or to "fail early," which is valuable if the program has put in place and funded 
contingencies. It also puts capabilities in the hands of the users more quickly. This 
incremental approach (as depicted in the pipeline discussion above) is the norm in 
commercial practice. 

Third, Congress should continue to support and refine programs such as the DoD’s 
Highly Qualified Experts Program and the IRS' Critical Pay Authority that help attract and 
retain critical government professionals. Additionally, IRS' pay-for-performance program 
has helped motivate performance aligned to outcomes. These are valuable tools that 
address the capacity, capabilities, and incentives needed to manage effective programs. 
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We encourage the Congress to look to these as models, streamline their execution and 
broaden their application government-wide— perhaps explicitly expanding them to 
include commercial IT Program Management expertise. 

Investing in people and establishing clear measures of success at the macro (program) 
level versus the micro (project) level have had positive effects in every case where we 
have seen this occur. Major IT programs are increasingly complex endeavors, and no 
matter how well organized, issues arise. The key is how one "rights the ship" when 
problems develop. Experienced and empowered leadership is an essential element. 

I request that my prepared statement be included in the record and I would be pleased 
to answer any questions. 
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Attachment 

Strengthening the Program Management Office 


An increasing number of today’s major government information technology (IT) acquisition 
programs are suffering cost and schedule overruns and/or failing to deliver usable capabilities. 
To improve success rates, Agency Chief Information Officers, the Office of Management and 
Budget, and Congress are searching for ways to ensure that government IT acquisitions are 
better planned and managed. In many failed programs, the government Program Management 
Office (PMO) did not recognize (or was unable to manage) common sources of program risk, 
such as unrealistic or unstable requirements. A technically strong PMO can improve the 
probability of program success by executing the disciplined systems engineering and program 
management processes necessary to manage risk effectively. 

The Importance of Strong Technical Capabilities 

To manage acquisition program execution successfully, the PMO must have strong technical 
and management capabilities. This begins with the Program Manager (PM). The most 
effective PMs possess proven leadership skills, mission knowledge, and understanding of the 
technical domain. The PM must understand the relationships among requirements, mission 
performance, cost, schedule, and risk. He must be empowered to make decisions that bring 
these competing elements into balance. The PM is accountable for program outcome within 
the authority and resources provided by the program sponsors. Even the most capable PM 
cannot succeed without access to (and support from) senior agency leadership. 

Supporting the PM in a technically strong PMO is a team that includes technical, mission, 
program management, cost estimation, and contracting experts. When staffed with the 
appropriate skills and experience, the PMO is capable of a peer relationship with the 
contractors) on technical and program management issues. Therefore, the PM is not forced to 
rely exclusively on the prime contractor for technical input to program decisions. The PMO 
team executes disciplined processes for program management, acquisition and contracting, 
and systems architecture and engineering. 

Program management 

Program management processes ensure that acquisitions are structured to deliver capabilities 
within budget and that program execution is managed to minimize risk while adapting to 
changing requirements and priorities. Program management processes include planning, 
assessment, control, decision-making, risk management, configuration management, and 
information management. 

Acquisition and contracting 

Acquisition processes ensure that qualified suppliers are selected and that agreements are 
negotiated with terms that, if fulfilled, ensure that the cost, schedule, and performance 
expectations will be met. Acquisition processes include developing acquisition strategies and 
plans, preparing solicitation packages, establishing evaluation criteria, evaluating responses to 
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requests for proposals, and evaluating vendor capability to meet technical requirements and 
stated commitments. 

Systems architecture and engineering 

Systems architecture and engineering processes ensure that systems are engineered to provide 
the desired capabilities within the constraints imposed by technology, available resources, 
schedule, external interfaces, operating environment, and regulatory requirements. These 
processes cover the full system life-cycle and include requirements definition, requirements 
analysis, architectural design, implementation, integration, verification, transition, validation, 
operation, maintenance, and disposal. 

Effective systems architecture and engineering is critical to acquisition program success. 

Figure 1 (Honour 2004) shows acquisition program cost and schedule performance as a 
function of the percent of actual program cost spent on systems engineering weighted by 
expert assessment of the quality of the systems engineering effort. Performance improves 
dramatically when a significant fraction (up to 12%) of program cost is for effective systems 
engineering. Today’s government IT acquisition programs rarely devote this percentage of 
program resources to systems engineering. 



Figure 1: The Value of Systems Engineering 


The “Architect" Function 

A key function of a strong PMO is best described by the metaphor of an Architect’s 
relationships with the user and the builder of a building. The Architect is the user’s agent and 
is independent of the builder. The Architect works to understand the user’s operational needs 
and translate them into technical requirements, defining the needed capability sufficiently to 
enable builders to develop it. The Architect evaluates development feasibility and performs an 
independent baseline design and cost estimate. These Architect functions enable the user to 
make informed cost/capability tradeoffs and prioritize requirements. The Architect is 
accountable to the user to ensure that the delivered capability meets the user’s highest-priority 
needs within the constraints imposed by available technology, funding, and time. The 
Architect accepts responsibility not just for the quality of his own work, but for ensuring 
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program success through orchestration of others’ contributions. Two of the most important of 
the Architect’s responsibilities are requirements management and conducting trade studies. 

Requirements management 

The Architect works with the user to understand operational capability needs and performs 
cost/schedule/capability tradeoffs to establish target system requirements that define the 
system sufficiently to enable one or more “builders” to develop the capability. Initially, the 
architect team works collaboratively with the users and other stakeholders to elicit and refine 
requirements. Tradeoff analyses and negotiations among key stakeholders are critical parts of 
this process. The result is an agreement on the prioritized set of system requirements that the 
system must satisfy. 

The initial requirements definition and tradeoff phase is rarely performed with sufficient rigor. 
In many agencies, responsibility for requirements definition, resource allocation, and 
acquisition are spread across multiple organizations without a process for making explicit 
tradeoffs among cost, schedule, and performance. The importance of spending sufficient time 
and resources in this initial phase cannot be overemphasized. Figure 2 (Gruhl 1992) shows 
acquisition program cost performance as a function of the amount of program cost spent on 
initial requirements and concept definition. Performance improves dramatically when a 
significant fraction (up to 15%) of total program cost is for requirements and concept 
definition. 

After system development begins, the Architect works with the system builders throughout the 
development to clarify requirements, perform design tradeoffs, and verify that the system 
design meets requirements. During this process, requirements may be revised due to changes 
in user needs, experience with selected technologies, or introduction of new technologies. If 
requirements must be revised, the Architect performs supporting analysis and negotiates 
requirements changes with the user. 
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Definition Phase as % of Total Estimate 

Figure 2: The Value of Initial Requirements and Concept Definition 
Trade studies 

The Architect performs analyses supporting PMO decisions throughout the program lifecycle, 
including cost and performance estimates, cost/schedule/performance tradeoffs, and 
evaluations of competing architectural and technical approaches. A “best practice” for 
performing these trade studies is for the Architect to develop and maintain a baseline system 
design and performance model throughout the program lifecycle. The baseline design is 
driven by technology as well as requirements; it is initially developed by the Architect, but 
evolves to reflect the contractor’s design. For the baseline design to be useful for trade studies, 
it must be detailed — sometimes down to the card level for hardware or the module level for 
software. With this level of detail it is possible to identify the critical technologies that need to 
be developed in the technology development phase and closely monitored during systems 
development. In addition, a detailed baseline design enables the PMO to generate accurate 
cost estimates. 

The nature and effectiveness of PMO trade studies can best be described by the following 
real-life case study of a program faced with selecting the best approach to modernize an aging 
computing infrastructure comprising processing, data storage, and database management 
capabilities: 

The system provides supercomputer-class processing power to users distributed over a 
wide geographic area and stores, maintains, and provides access to enormous quantities 
of data. The legacy capability was a centralized processing and data storage architecture 
built from proprietary hardware and software. It was obsolete and expensive to maintain. 

The goals of the modernization program were to improve performance and reduce 
maintenance costs while minimizing impact on users’ business processes. In addition, 
transition from the existing to the replacement system had to be performed with no 
disruption to the users. 

Performing the architectural trade study presented the PMO team with several challenges. 

The user requirements were poorly defined and their business processes were not 
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documented. The quantitative requirements that were known (e.g., throughput, memory, 
bandwidth) were valid only for the legacy architecture. In addition, the incumbent 
contractor asserted that refreshing the existing architecture with new proprietary 
hardware was the only viable option. They based their conclusion on an analysis that 
appeared to understate the performance of other architectures. Finally, there were many 
possible architectural alternatives, which created a large trade space. Options ranged from 
replicating the existing centralized supercomputer architecture to developing a new fully- 
distributed cluster-computing architecture. 

The PMO team conducted a comprehensive review and documented the users’ business 
processes. Based on the business process description, the team developed architecture- 
independent system requirements. The PMO team’s conflict-of-interest-free status gave 
them access to proprietary vendor data, including case study data from operational cluster 
computing installations. Based on the derived system requirements and the data collected 
from potential vendors, the team synthesized five complete baseline designs covering the 
entire architecture trade space and developed an architecture-independent system 
performance model. Using the baseline designs and performance model, the team 
generated life-cycle cost estimates (including the cost of future upgrades) for each 
baseline design. Finally the team conducted a risk assessment focusing on potential 
disruption to mission-critical activities during the upgrade. 

The results of the trade study indicated that there were only modest life-cycle cost 
differences among the architectural options. A centralized architecture minimized data- 
transport costs, while a fully-distributed architecture minimized hardware costs. 

However, a hybrid option that distributed some functions while centralizing others 
provided a 20% increase in user productivity. Based on the study results, the program 
selected a hybrid architecture option. The new system was developed, and the transition 
was made seamlessly within projected cost and schedule. Furthermore, a user 
productivity advantage was realized from the new system consistent with the study 
projections. 

Conclusion 

Today’s government IT acquisition programs are executed in a complex, uncertain 
environment. Rapidly evolving roles and missions create requirements volatility, and growing 
operational interdependence of organizations increases the number of program stakeholders 
and dependencies. In many failed programs, the government PMO’s inability to manage this 
uncertainty and risk resulted in a failure to meet cost, schedule, and performance expectations. 

A technically strong PMO provides an “Architect” function that enables the government to 
make informed decisions and manage the increased risks in today’s environment of 
uncertainty, improving the likelihood of success in complex IT acquisitions. Unfortunately, 
the government is suffering from years of erosion in its organic ability to perform the 
functions of a strong PMO, particularly the Architect function. An aging workforce, difficulty 
in attracting new talent, and an explicit strategy to reduce the size (and expense) of PMOs are 
the root causes of the erosion and will be difficult to reverse. However, the Architect function 
can be performed outside of government as long as the Architect is sufficiently free of 
conflict-of-interest to serve as the government’s trusted agent. 
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Actions to Avoid “IT” Train-wrecks: An Agenda for Change 

Good morning, Chairman Carper, Ranking Member Coburn, and distinguished 
members. First, let me thank you for inviting me to testify, and to congratulate you for 
holding this hearing, since literally billions of taxpayer dollars go down the drain every 
year in both visible and invisible Information Technology (IT) acquisition waste, and 
clearly you are on to something important. Vice Admiral Jerry 0. Tuttle (RET) , the 
former Deputy Chief of Naval Operations for C4I and icon for naval computing and net- 
centric warfare, would counsel “Lead, Follow, or Get out of the way”; thank you, 
Senators, for leading the way. 

IT projects too often experience problems of Cost Explosions, Schedule Black-Holes, 
Performance Disappearances, and large-scale Train-wrecks — many caused by violating 
one or more fundamental laws of “IT” Physics, described later in my testimony. 
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Let us be clear about one thing: Although effectively managing a large-scale IT program 
is difficult, on the other hand, producing a large-scale IT Train Wreck is easy; the good 
news is that Wrecks can be avoided by effectively using well-known practices. 

Today I would like to briefly offer actions that government Departments, Agencies, 

OMB, and Congress can take to prevent them. At its core, these actions address rapidly 
achievable improvements. 

With your permission I will summarize my testimony and submit it in full for the record. 

I’ll begin with a structural observation: Much is expected of each Agency CIO. Many 
have responsibility without real authority. Many Federal Departments include 
numerous, essentially independent “fiefdoms” because congress has so arranged it, 
fiefdoms independently funded by Congress. The Pentagon rule is, “He that’s got the 
gold makes the Rules’’— so too in Federal Departments. Although I don’t today have 
any solutions to offer, I would be happy to work with your staff to perhaps devise some. 

Next an observation'regarding those IT problems as relating to project management and 
oversight: When OMB testified before you last September, they expressed their recent 
interest in IT program execution, and that’s a very good thing; but IT programs don’t 
manage themselves. 

From my understanding, very few Agencies have much in the way of any real IT 
program management and oversight. Earned Value Management is held-up as the do- 
all silver-bullet solution, yet little is done to prevent the easy gaming and corruption that 
Earned Value is vulnerable to, and associated rebaselining may lack the transparency 
needed to ensure effective oversight, and far too much is expected of it. Although it’s a 
powerful visibility technique that supports program management, it cannot replace 
program management. Unfortunately, there seems little in the land of government IT 
Program Management that implements needed essential techniques like Risk 
Management, Requirements Management, or Integrated Baseline Reviews. 

As a solution, I would recommend that each Agency be required to create their own 
“process lite” version of IT program management and oversight; that their focus be on 
implementing the critical detail that makes these processes effective with minimum 
overhead; and to use automated tools to identify remaining weaknesses and 
vulnerabilities. And while they are at it, it would be a good time to apply the “Lean Six 
Sigma” waste-cutting improvement process across OMB and the Agencies — to chop-off 
potential nodes of unnecessary work and all other non-value-added activities built-up 
over time in attempt to inspect-in quality and defuse responsibility. 
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Transparency is dandy, but it must be converted to visibility to be useful. A 
“transparent" contractor can deliver a 53’ truck full of boxes of data, but what you really 
need is only the bottom-line information. What’s needed is true visibility of IT project 
health and progress — in near real-time. 

Since Earned Value seems to be the only principal visibility technique relied on by 
OMB, I would propose a more comprehensive visibility product — Exhibit 350 — to 
provide real project visibility indicators monthly, primarily for the Program Manager, 
with quarterly simplified versions for component and Agency CIOs, OMB and Congress. 
I’ll be happy to work with your staff and OMB to flesh this out if it would be useful. 

Tracking schedule progress is not easy, and requires at least a good Task Activity 
Network. One reason for schedule surprises is that as pressure to meet schedule 
increases, the “hard-to-do things” are “kicked-down-the-road”, with uncompleted 
difficult work now moved into future builds. As it turns out, these future builds typically 
will require a successive series of miracles to be accomplished in-order to complete the 
development on time and on budget; don’t bet on those miracles happening. 

To motivate Agencies to focus on IT project management and oversight, I would 
recommend adapting the “Nunn-McCurdy” notification process. Not that it’s a great 
visibility technique — its not. But it in fact serves as a powerful motivator as something 
really to be avoided; as little else can do, it gives clear focus to the business of cost and 
schedule containment. 

We have a serious problem regarding people: It’s difficult to reward good talent, to hire 
good talent, to train good talent. We expect CIOs and our IT personnel to do more with 
less and then give them less, as if to prove the point. OMB needs to address this as a 
priority. 

Training of project personnel in effectively implementing fundamental processes is 
minimal; while DoD has a certificate program in IT Program Management, GSA has 
long ago disbanded their excellent similar “Trail Boss” program for civilian agencies. A 
lack of training for the various communities charged with making IT developments 
work— the requirements, resourcing, testing and sustainment communities are even 
further behind management training — those communities have received essentially no 
education, training or any certification to do what they are asked to do. No training; no 
education; no certification and no experience-it's a wonder we do as well as we do, as bad 
as it is. 

A Program Termination Process: The former Assistant Secretary of the Army for 
Acquisition, the Honorable Claude Bolton, is a strong proponent of terminating 
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projects if measurable outcomes cannot be achieved within the agreed-to program 
cost, schedule, performance baselines. The DoD has no formal decision making 
process or policies to terminate programs, and I am not aware of any for other Agencies. 

I think it wise and prudent to consider including such a process among ways to improve 
the government’s IT acquisition process. 

Contract incentives and other considerations are important, and I discuss them further 
in my testimony. 

Now I will address my concerns and recommendations in more detail: 

PROGRAM CONTROL AND MANAGEMENT 

First, Require Effective Implementation of Fundamental Processes. 

Congress could and should require Departments and Agencies to formalize their 
processes for risk management, visibility and metrics, and other fundamental 
processes — acquisition planning, requirements development and management, 
continuous estimation of cost and schedule, and program management and 
oversight — together with mechanisms to assure their effective use and 
accountability, identification of their weaknesses and vulnerabilities, and their 
continuous improvement. 

Focus on Rapid Deployment of Critical Detail. “Process Improvement” can come with 
a lot of baggage; Process-Priests make it into a religion; but focused on a core of 
effectively-implemented critical detail, fundamental processes can make a big 
difference in project success. Implementation of fundamental processes need not 
be overly complicated: pithy descriptions of actionable critical-detail coupled with 
quick deployment is infinitely more preferable to creation of study groups and 
“process action teams” who extensively coordinate, plan and define 
comprehensive lengthy descriptions which ultimately are never implemented; 
start with the core-details that are critical for a process to achieve its bottom-line 
potential and rapidly deploy them; avoid "box-checking” of label-engineered 
processes that produce great-sounding but hollow processes which — (borrowing 
from Macbeth (Scene 5)) , have a sound and furry of correct-titles but whose 
actions signify nothing. 

An Annual Agency IT Report to Congress should already be in the mail. Having seen 
from the inside the power of a simple annual report to Congress to bring-about 
real change in government, I would earnestly recommend such a report — from 
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each Department Secretary and Agency head — covering IT project status and 
health, and also progress in implementing fundamental process improvement. 

Risk Management is the first fundamental process. Although all of these fundamental 
processes are equal in importance, Risk Management is first among them, and 
should be first deployed and checked for effectiveness. 

Independent Expert Reviews are an important adjunct. The Defense Science Board 
study on software has recommended projects be reviewed by a team of 
independent experts because they provide an essential and needed means for 
identifying project risk. Despite its powerful potential for avoiding IT Train- 
wrecks, its use in government projects is, unfortunately, rare. 

Require Certification of the Requirements-Budget Estimate. There are a number of 
powerful techniques that support the definition, management, tracking, and 
related cost estimation of requirements, which can and should be applied to ensure 
that project cost and schedule estimates are current and accurate. OMB could 
require that Exhibit 300’s include a certification (together with supporting 
information) that the budget estimation process results in a realistic cost and 
schedule estimate for implementing existing requirements and addresses potential 
requirements growth. The certification should also describe the specific detailed 
process for requirements definition and cost/schedule estimation techniques 
which were utilized. 


VISIBILITY 

Transparency is lacking — missing is provision of relevant known, program status 
information to all stakeholders: Notwithstanding OMB’s new and important focus 
on management of project execution, OMB’s Management-Watch-List and High- 
Risk-List, while valuable adjuncts, do not provide the additional transparency 
needed to anticipate and identify emerging problems. 

Conversely, Visibility is needed. Making the important aspects of project status and 
progress known is paramount. As the Sixth law of IT Physics states, “You can’t 
manage what you can’t see.” There are a number of important metrics and 
indicators that are needed to provide information essential to effective and 
prudent management of large-scale IT projects, with differing levels of detail 
needed for Project Managers, Department or Agency management and senior 
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leadership, OMB, and Congress. Visibility becomes more useful as things become 
more tangible: Quantative measures; Knowledge Points; and frequent tangible 
deliveries of working product. 

A new Exhibit 350 would go a long way. OMB could devise a much needed visibility 
companion to its Exhibit 300 documentation, which I denote as Exhibit 350 to 
indicate its intertwined relationship with its companion Exhibit 300. Exhibit 350 
would constitute the core of a status report produced for and reviewed monthly by 
the project manager for each OMB-approved program, and a less detailed variant, 
say Exhibit 350-A to be provided quarterly through the cognizant Department or 
Agency Officials to OMB and Congress. Exhibit 350 would include a number of 
current indicators of project health and progress, such as: predicted Functional 
Availability; Defect Status; Integration Progress; Currently Estimated Cost at 
Completion and other important Earned Value indicators; Currently Estimated 
Schedule Completion, and Earned Schedule; Defect Containment; identification 
of key risks and status of associated Mitigation Activities; and other indicators 
tuned to the project’s current phase. 

Earned Value Management. While Earned Value, which quantifies products by 
intermediate and final products produced is a fundamentally important visibility 
process, however, it can be readily gamed or corrupted, producing misleading 
results which can be more dangerous than not implementing EV, since 
unrealistically positive results generate complacency. Seriously flawed Earned 
Value may be difficult to detect; however, experts can ferret-out the gaming and 
commonly made “adjustments” that can produce spurious and misleading results 
leading managers astray. Two of the common easily detected corrupting 
influences are tasks without products, and rubber-band rebaslining. While a few 
IT projects scan their Earned Value implementation for such landmines, most do 
not. 

Nunn-McCurdy Certification. Nunn-McCurdy is a powerful visibility technique 
which should be borrowed from the Congressional Defense oversight process for 
government IT programs. Such a certification would require a Department 
Secretary or Agency Head, upon a breach in excess of some specified amount, say 
25% of the contract cost baseline, to certify to Congress the continuing need; 
causes of the breach; that effective corrective action was put in place to prevent a 
future breach (including description of such actions) ; and, that a review of other 
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major projects has been conducted to check for potential existence of similar 
problems. Perhaps a similar certification might be considered for similar schedule 
slips. Although not a predictive mechanism (since its “red-light’ goes on only after 
a significant cost increase has just about, if not already occurred), its greatest 
benefit is as a highly effective motivator throughout every Department and 
Agency — since it focuses attention at every associated management and executive 
level (including congress) on actions that could have been taken to avoid such a 
certification, and creates uncertainty as to potential negative effect upon 
congressional appropriations. 


Incentives 

Contract Incentives. When properly employed, the contractor’s interest and 
government’s interest become aligned, cooperation and shared risk exist, and the 
likelihood of a successful outcome is significantly enhanced. Serious focus by 
agencies on contract incentive structures can make dramatic improvements in 
successful IT project outcome. The trick is creating the proper incentives-that’s 
critical-but the details of this constitute an entire other discussion. 

Employee Incentives. People. Teams. Success. People are more important than 
any process. Much has been written about the success of large-scale IT project 
development as dependent on knowledgeable, dedicated people and effective 
teamwork. Government personnel rules do little to encourage and incentivize the 
career government IT expert. Little meaningful financial incentive exists for 
outstanding contribution; in one Department, rewarding such a federal employee 
with a five percent bonus requires the personal approval of the Secretary of the 
Department. There is little likelihood that such a request would be made, and the 
betting is that it would not be approved. The government’s ability to reward and 
incentivize its employees is in sorry shape, and OMB should initiate coordinated 
action to address this issue. The same can be said for the difficult conditions faced 
by CIOs and IT project managers with regard to their ability to hire needed 
talented employees. We ask them all to “do more with less”, then we actually give 
them less, and are surprised at resulting failures. 

Training of project personnel in effectively applying fundamental processes is 
virtually absent. These diverse activities include rapidly deploying an effective 
Earned-Value and Risk-Management process, establishing meaningful project and 
agency metrics, determining effective test strategies, structuring contract 
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incentives, managing and tracking requirements, maintaining a good cost and 
schedule estimate, using simulation to predict project completion within specified 
confidence limits, identifying risks, and ensuring a solid Task Activity Network. 

* * * * * 


The Laws of IT Physics™: Although there are many factors influencing a 
successful project development outcome, and other “Laws" and corollaries exist, 
the following have been selected as most relevant to this hearing and testimony. In 
the interest of brevity, substantial explanatory text associated with these laws and 
their corollaries has been omit ted. 


First taw 

Planning is a continuous process, not a ane-time event. 


A Project Plan cannot survive past contract award and must continually change 
based on actual experience {requirement additions or modifications count as actual 
experience). 


Second taw 

Complexity kills IT projects since defects and security vulnerabilities increase 
noniineariy with increased complexity. 

Corollary 

Minimizing and controlling complexity are key to successfully achieving a large-scale 
system development success both in the development of individual releases and in 
the cost and schedule of downstream upgrades to operational software. 

Government Project Offices should ensure that the complexity of system architecture, 
each lower level of design, source-code, and task activity Networks is minimized. 


Third Law 

Schedules and project chaos create Event Horizons, from which a project cannot 
recover. 

Corollary 

Avoid the Project Event Horizon; Compute Schedule Compression and Monte-Carlo 
simulate the Task Activity Network. 



The initial requirements for any large system will be incomplete. Independent of the 
resources expended to develop them. 

Ensure planned requirements can be delivered within cost and schedule estimates, 
but also include budget for anticipated and actual requirements change; rigorously 
test, accept, and track requirements as they are met. 


Fifth Law 

Unvalidated requirements pave the road to project failure. 

Corollary 

Test and validate requirements as early as possible before basing significant 
projects upon them; use pilots where possible before fully committing. 
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1 Sixth Law 


You can’t manage what you can’t see. 

Corollary 

Track Project Status and Progress against small, testable, incremental product 
deliverables and use quantitative project parameters, such as Earned Value, to 
make projects visible and manageable. 


pp 
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Not controlling the right things assures failure. 

Use well established best practices such as Risk Management, Requirements 

gemenf; Defect Management; and integrated Baseline Reviews to control 



<ghth law 

Poor defect management causes high rework and leads to project failure. 

Corollary 

Use automated testing and continuous integration to prevent defects, and j 

continuously identify out of phase defects to correct their root causes. 


Ninth law 

Unknown and untreated vulnerabilities originating in ineffectually implemented 
Processes destroy IT projects. j 

Corollary 

Automate vulnerability identification and prioritize fixes which root-out and fix 
processes lacking critical essential detail needed to achieve bottom-line objectives. ; 


Tenth law 

Development Contractors wifi do what is in their financial interest, and government 
organizations may be led toward a project Event Horizon. 

Corollary 

incentivize well and wisely, trust but verify, and use Award-Fee type contracts; 
carefully construct the Award-Fee criteria to address principal project objectives 
over the near term; identify what Award-Fee structure will sufficiently motivate the 
development contractor. 


Eleventh Law 

Thoughtful, knowledgeable, committed people operating as a team are critical to IT 
Project Success. 

Corollary 

Trecn people as the valuable resources that they are; take actions to create and j 

maintain “jelled” teams. 
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Chairman Carper, Ranking Member Cobum, and other members of the Subcommittee, I 
thank you for the opportunity to speak with you today. As Secretary of Delaware’s Department 
of Technology and Information (DTI), I can well appreciate the complexity, the challenges, and 
the significant responsibilities associated with managing information technology projects in an 
investments portfolio that cuts across many agencies, and in Delaware’s case, all three branches 
of government. Albeit on a much smaller scale than the federal government, Delaware (and other 
state governments) are faced with similar concerns regarding IT project management. In fact, 
Delaware’s Department of Technology and Information was established in part because of the 
ongoing IT project delays and cost overruns that Delaware was experiencing. Delaware’s 
centralized IT structure charges our agency with direct oversight and approval of nearly $200 
million in active IT projects. 

As a new agency we were able to develop and deploy new approaches to IT project 
management that incorporate many best practices from private industry and others in government 
sector. While our methods are under constant review for improvement, we are enjoying some 
significant progress and we have an excellent track record of delivering much needed IT 
solutions that are on-time and on-budget. 

There are no “silver bullets,” no one or two changes that you can point to for perfect 
project management. Instead, there are many small improvements that we have made that, in the 
aggregate, are making a difference for Delaware. I would like to take a few minutes to give you a 
high-level view of a project lifecycle using the “Delaware Model.” In addition, I would like to 
point out what I believe are the significant processes and procedures that we have put in place to 
successfully manage our portfolio. 

Like the federal model, we require agencies to submit a business case that addresses the 
major items we believe help insure a project’s success. Our model includes the following major 
areas: risk management, processes re-engineering, architecture review, resource and funding 
availability, project management oversight, organizational change management, needs 
assessment, customization requirements, disaster recovery levels and management/executive 
sponsorship. 

Recommended projects are forwarded for the concurrence of the State CIO before they 
move ahead in our process. This recommendation usually includes ongoing funding contingent 
on meeting project milestones. 

Once approved, DTI works with the agency-customer to develop full and complete 
requirements so a request for proposals can be released to secure vendor bids in order to meet the 
needs of the project. DTI stays involved to help the agency make a vendor selection, and to 
structure a contract that ensures the project is delivered on-time and on-budget. I cannot 
emphasize enough how critical the requirements gathering process is to the project’s ultimate 
success or failure. The Delaware Model does not allow for requirements gathering to be 
conducted solely by the vendor, there must be active involvement by the DTI project 
management team. There is an old saying, “the customer doesn’t know what they want, until you 
give them what they asked for.” Requirement gathering is critical to the process and helps 
alleviate scope, time, and budget creep if done correctly. 
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At the start of a new project, a nationally certified program/project manager is assigned 
as well as a certified organizational change management team to run parallel courses in 
managing the projects. It is not enough to be certified, we take extra care in selecting the people 
to fill these positions as they are essential to the success of the project. We count on these folks 
to inflict discipline and structure to our projects. 

It is important to note that DTI manages Delaware’s IT projects, not the vendors assigned 
to them. While we demand the vendor assign a certified and talented project manager, our state 
folks manage the project. Further, I would like to highlight our change management process. 
Change management starts with the review of existing business processes and is focused on 
preparing the organization for the cultural changes that are a part of any major projects. 
According to a recent study conducted by the National Association of Chief Information 
Officers, which represents state CIOs and Information Technology Executives from the states, 80 
percent of major IT project failures can be directly attributed to a lack of change management. 
Too often, the employees who will actually use the new application are left out of the project 
process and when this takes place it is almost certain that the project is doomed to fail or 
underperform. 

Another major element of our success is how we manage the overall process. All of 
Delaware’s major IT projects have executive sponsors. The executive sponsors include high- 
level managers from the agencies involved, as well as, key executives from DTI, and our Office 
of Management and Budget. The executive sponsors hold regular monitoring and update 
meetings and provide high level oversight. When difficult project decisions need to be made, the 
executive sponsors are aware of the issues and have the authority to make critical decisions on 
whether or not to keep the project moving along. 

A real-life example is our present project involving all of Delaware’s courts. Due to a 
lack of employee training and readiness, the executive sponsors recommended that the project be 
“paused,” so that training could take place. In fact, Delaware’s Chief Justice himself invoked a 
six month pause in recognition of the need for court employees to be prepared if the project was 
to succeed. 

Besides ongoing project meetings, all projects in the State’s portfolio are reviewed 
weekly by our project management office and bi-monthly at a workload management meeting 
where project managers present their projects’ status to the assembled DTI senior managers. 
Opportunities for improvement or needed adjustments are vetted in an open forum with the goal 
of keeping the project on track and transparent. 

Our agency is committed to project transparency, all the way to our legislature and our 
governor. We believe in continuous information exchange and dialogue with our elected 
representatives so that they are educated and aware of the complexities of major IT projects. 
Providing information upfront, even when it may be painful, is far better than “saving” 
unpleasant surprises for yearly budget sessions. Bad news doesn’t age well. 
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I wish I could tell you that everything is humming along perfectly and that the processes 
that we have put in place are the final answer. But I can’t, as we have many challengers similar 
to those being talked about today. However, we believe that by tightly managing these 
challenges we can deliver projects on-time and on-budget. Although our project tracking system 
is complex, in our communication with stakeholders, we use high level red, green, yellow 
scorecards to inform them of how the projects are developing. 

All projects have, or will have, ongoing issues that require senior management attention. 
Under the Delaware Model, we believe that providing the facts, good or bad, to all stakeholders 
- including our legislature - is the proper policy, and most importantly, lives up to one of DTI’s 
core values, “Integrity.” Our vendors are held accountable to deliver what they said they would 
and we call the facts as they are. 

Thank you for the opportunity to testify today and I would be happy to answer any 
questions. 
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Post-Hearing Questions for the Record 

“Offline and Off-budget: The Dismal State of IT Planning in the Federal Government” 

July 31, 2008 

Questions for the Record from Senator Thomas R. Carper to the Honorable Paul Denett 1 

1 . Mr. Denett, it seems you have acknowledged that OMB has dropped the ball when it 
comes to reporting to Congress on major capital assets as required by the Federal 
Acquisition Streamlining Act (FAS A) of 1994. Although it is unfortunate that this 
mistake was caught on your watch, it is something that has not happened for some time 
and needs to be addressed before the next presidential transition. 

a. What specific steps have you taken to make sure your office will continue to 
report this information to Congress annually? 

Response : The Office of Federal Procurement Policy (OFPP) is working 
with the Resource Management Offices (RMOs) in the Office of Management 
and Budget (OMB) to determine the best way to collect, analyze, and report 
on non-information technology (IT) investments. OFPP is also consulting 
with the Project Management Working Group (PMWG), which was formed 
by the Chief Acquisition Officers Council late last year to help improve the 
use of capital planning and project management for major acquisitions. 

b. And why do you think there was a lapse in reporting in the first place? 

Response : Comprehensive information covering the same issues required by 
the reporting requirement in title V of the Federal Acquisition Streamlining 
Act (“FASA V”) — including cost, schedule, and performance information — 
has been provided annually to Congress through Chapter 9 of the Analytical 
Perspectives volume of the President’s Budget. It was presumed that this 
information was addressing the FASA V reporting requirements, at least 
with respect to IT investments, which has been the focus of Congressional 
interest, including that of this Committee. 

2. Historically, this hearing has focused solely on the proper planning and execution of IT 
investments. However when my staff was investigating OMB’s oversight role for all 
capital assets, of which IT is a subset, it seems that OMB does not collect, analyze, or 
provide the same type of information to Congress that the Office of E-Govemment and 
Information Technology does for IT. 


1 The Honorable Paul Denett resigned as Administrator of OFPP on September 2, 2008. The responses in this 
document were prepared by the OFPP staff. 
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a. What type of investment specific information does your office collect 
government-wide on capital assets besides IT? 

Response : OMB Circular A-ll requires agencies to develop business cases 
for their major non-IT capital assets used by the Federal Government and 
having an estimated useful life of two years or more. The business cases 
include information covering project management, acquisition strategy, 
performance information, security, privacy, enterprise architecture, 
alternative analysis, risk management, and cost, schedule and performance. 

Agencies work with OMB to identify which new and ongoing acquisition 
programs require closer evaluation and should be reported to OMB, taking 
into account factors such as importance to agency mission, significance in 
terms of program or policy implications, high executive visibility, or high 
development, operating, or maintenance costs. There currently is no 
uniform government-wide portfolio assessment of non-IT capital asset classes 
similar to that used for IT, in part because major acquisition programs 
outside of IT take a variety of forms, ranging from construction and real 
property to aircraft acquisitions. OFPP is consulting with the PMWG to 
evaluate how performance-based management for different types of major 
non-IT investments is best applied. 

b. In what format do you collect this information? 

Response : Circular A-ll requires agencies to submit business cases using 
the format prescribed in Exhibit 300. The Exhibit 300 requires agencies to 
provide specific information on performance goals and measures, results 
achieved against goals, acquisition strategies, and project management. 

c. How often is this information updated by the agency? 

Response : Under A-ll, agencies are expected to update Exhibit 300s to 
reflect decisions in the Budget. If agencies request supplemental funds, 
which include changes to the agency's portfolio, agencies would need to 
update their Exhibit 300(s), as appropriate. Agencies are expected to 
continuously maintain updated information about the projects contained in 
the Exhibit 300. 

d. What type of analysis does your office perform to make sure the agency is 
spending the money wisely? 

Response : OFPP works with the RMOs during the development of the 
budget to support the review of the business cases that have been submitted 
to OMB. OMB uses the agency Exhibit 300 submission to make both 
quantitative decisions about budgetary resources, consistent with the 
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Administration’s program priorities, and qualitative assessments about 
whether the agency’s programming processes are consistent with OMB 
policy and guidance. 

To assess agency progress in implementing capital programming principles 
and practices for non-IT investments, OFPP directed agencies to provide 
information on their new and ongoing non-IT major acquisition projects. 
During its assessment, OFPP considered whether agencies: (i) have capital 
planning and investment control policies for major acquisitions other than 
IT, (ii) have established cost, schedule, and performance goals for new and 
ongoing projects, (iii) plan to use Performance-Based Management Systems 
(PBMS) to monitor acquisition progress on new programs, and (iv) are using 
PMBS to monitor ongoing program progress. The results of OFPP’s 
assessment were reported to Congress in its FASA V Report, dated July 30, 
2008. 

e. And how does your office work with the agency and with Congress to make sure 
we are proper fiscal stewards? 

Response : The general principles for non-IT investments are the same as 
those for IT. As OMB’s Capital Programming Guide explains, when an 
agency reports that a major acquisition is not projected to achieve 90 percent 
of goals, OMB works with the agency on remediation activities, reviews the 
agency’s reasons for deviation from goals, the reasonableness of the 
corrective actions proposed, and the validity of increased cost estimates. 

In addition, OMB and the Government Accountability Office (GAO) have 
worked with the Department of Defense, the Department of Energy, and the 
National Aeronautics and Space Administration to mitigate risk in capital 
investments by (1) establishing clear definitions of success and (2) developing 
corrective action plans with milestones and metrics that hold agencies 
accountable for meeting the goals that have been collectively agreed upon for 
the successful attainment of cost, schedule, and performance goals. 

f. Lastly, do you believe the information you provide Congress allows us to spot 
problems in non-IT capital assets before they spiral out of control? If so, how? 

Response : Agency use of performance-based management for non-IT 
acquisition programs has generally been more limited than that for IT 
acquisitions, which has limited the information that is readily available on 
non-IT capital assets. OMB is committed to working with agencies to 
improve these practices so that better information is available to identify and 
address project shortcomings before taxpayer resources are wasted. 
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3. Like Ms. Evans, your office oversees billions of dollars that could potentially be wasted, 
abused, or even stolen. However, I question whether OMB is providing the proper 
oversight that is needed to hold agencies accountable when an investment becomes a 
money-pit. 

a. Will you commit to providing Congress a single, aggregate list for all major 
capital assets that tracks their cost, schedule, and performance from the 
investment’s inception to its completion? 

b. If so, when will this information be provided? 

c. If not, why not, and what do we in Congress need to do to make sure we properly 
oversee the hundreds of billions of dollars that is spent on non-IT capital assets? 

Response : As OFPP’s FASA V report explained, capital planning and 
investment control policies are generally not as well established for non-IT 
projects as they are for IT investments and performance-based management 
systems are not always being used to track cost, schedule, and performance. 
OFPP is reviewing, in consultation with the Chief Acquisition Officers, how 
more complete information can be collected in order to improve 
transparency and increase accountability. OMB looks forward to working 
collaboratively with Congress to improve upon the ways in which we deliver 
and share information. 
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Post-Hearing Questions for the Record 
Submitted to Karen S. Evans 
From Senator Thomas R. Carper 

“Offline and Off-budget: The Dismal State of IT Planning in the Federal Government” 

July 31, 2008 


Question: 

1. Ms. Evans, this Subcommittee had enjoyed a wonderful working relationship with you since 
you first testified before us two years ago. As I have said before, I must thank you for all of the 
progress that has been made in the area of IT planning and management. However, it doesn’t 
mean that there isn’t more work we can do. Specifically, I still do not believe Congress has the 
information required to hold agencies accountable — which in large part is our job - to oversee 
and track an investment’s cost, schedule, and performance information from its inception to its 
completion. Most of these projects span several years and are worth millions; yet Congress only 
recently learned a month ago that some of these projects have been poorly planned for over 2 
years. 

a. Do you believe that OMB provides Congress the information necessary to track the 
performance of an IT investment from its inception to its completion on measures such as cost, 
schedule, and performance goals? 

Response: 

In recent years, OMB has taken steps to provide Congress direct information or the ability for 
Congress to access agency information on the planning and execution of IT investments 
throughout their lifecycle. The steps have included the following: 

* Quarterly reporting of the Management Watch-list 
( http://www.whitehouse.gov/omb/egov/documents/MEtWatchList.pdf ) 

* High Risk-list 

( http://www.whitehouse.gov/omb/egov/documents/High Risk List 03FYQ8.pdf ) 

■ Exhibit 300 Evaluations Profile 

(http://www.whitehouse.gov/omb/egov/documents/Exhibit300 Evaluations Profiles. 
pdf). 

Additionally, OMB has enacted policy changes through OMB policy memo ( M-05-23 ) to enable 
Congress (through GAO) and agency Inspectors General to access agency performance data for 
all high risk projects not meeting one of the four principal criteria as provided in the policy 
memo. 
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Lastly, OMB modified Circular A-ll, Section 300.7 in 2007 to require agencies to post public 
versions of their Exhibit 300s to their website within two weeks of the release of the President’s 
Budget. 

The processes currently in place at OMB for evaluating potential and active investments in IT 
are not designed to provide real-time project performance data. For those projects that 
experience critical issues in between quarterly reporting processes, agencies typically notify 
OMB on an interim basis. In the case of the Census project, the issues involved were not 
isolated to IT. 

b. If you feel the information is adequate, how come Congress was caught off-guard with several 
investments that have spiraled out of control and required Congress to pass emergency funds; 
such is the case with the Census? 

Response: 

OMB is in the process of providing a user friendly way to navigate through the Federal Report 
on IT Spend and will identify both Management Watch List (MWL) and High Risk List (HRL) 
investments along with the rationale for an investments’ inclusion on the MWL or HRL, and 
where possible, identify multi-year members of the MWL. The data will be downloadable to 
allow users to conduct further analysis and print reports. 

The goal of this new functionality is to make the underlying data more useful and easier to 
access. The portal has the added benefit of improving data quality by presenting information in 
context to both authors and consumers of the information. 

c. If you do not feel the infonnation is adequate, what information will be helpful to Congress so 
we can track investments government-wide and prevent future problems before they spiral out of 
control? 

Response: 

OMB will continue to evaluate potential investments in information technology against several 
strategic and management criteria as part of the budget process, including identifying those that 
demonstrate a moderate to high likelihood to encounter project management and execution 
problems; however, instituting a simplified reporting process for in-process acquisitions that are 
mission critical, high profile or dollar value, or are otherwise at risk, using Earned Value or 
comparable infonnation would provide both OMB and Congress with current information from 
which to take preventive action. 

Also, OMB encourages Congress to use the functionality discussed above, provide feedback and 
continue to work with the E-Gov Office. 

d. Lastly, why hasn’t OMB provided Congress a single, aggregate list of all IT investments that 
track whether they are poorly planned; how long they have been poorly planned; the reasons why 
an investment is poorly planned; if the investment is achieving cost, schedule, and performance 
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benchmarks, and if not, what percent the investment is off; whether the agency is properly 
tracking the investment with Eamed-Value Management; and whether the investment has been 
rebaselined, and how many times it has been rebaselined? Isn’t this all important information for 
Congress to know? 

Response: 

OMB looks forward to working collaborativeiy with Congress to continue improving upon the 
ways in which we deliver and share information. The portal described above is a step forward 
and potentially provides a platform to streamline and improve the performance information we 
provide publically and to Congress in the coming year. 

Question: 

2. You stated in an earlier response that you didn’t feel it was OMB’s responsibility to embarrass 
an agency, but instead help them to correct problems. I agree with this statement, only to the 
extent that someone needs to be holding these agencies accountable, and I don’t feel that is 
happening now at OMB. Furthermore, you mentioned that instead of Congress focusing solely 
on those investments that are doing poorly, to instead focus on those investments that are doing 
exceptionally well. Once again, I agree with this statement, only to the extent that OMB needs to 
provide Congress the information necessary to hold those agencies accountable that are doing 
poorly, but also provide the information necessary to point out those agencies as models that are 
doing well. 

a. Ms. Evans, how can the American people trust that OMB doesn’t have a conflict of interest 
when helping agencies to right problems, but holding them accountable when money shouldn’t 
be spent on bad projects? 

Response: 

OMB’s charge and interest is to provide oversight and policy direction to agencies. This 
includes working with them to achieve success, as well as putting controls in place to ensure 
proper management. OMB works with agencies to correct program deficiencies and to be good 
stewards of public resources. OMB holds agencies accountable through the President’s 
Management Agenda (PMA) Scorecard quarterly process, public reporting of the Management 
Watch List and High Risk List, and analysis of IT investments during the budget formulation 
process. In addition, OMB holds monthly or quarterly briefings with agencies on critical and/or 
struggling IT projects. 


b. Further, can you provide this Subcommittee any and all times your office has held an agency 
accountable through the Apportionment B process since 2001 for an IT investment and the 
reasons why OMB redirected these funds? 

Response: 
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OMB uses a wide variety of management mechanisms to ensure agencies are held accountable 
for their performance as it relates to their IT investments. Agencies provide OMB with regular 
updates on agency projects, including through the PMA Scorecard process on a quarterly basis, 
through agency budget exhibits, and through other annual reporting. Through these processes, 
both the agency and OMB are able to identify investments that need additional focus to ensure a 
successful outcome. In that regard, OMB has also utilized its Clinger-Cohen authority to ensure 
agencies are accountable for their investments by providing specific funding levels in their 
apportionments. 

The Internal Revenue Service Business Systems Modernization (BSM) account was apportioned 
by category B in the past to ensure project-by-project spending was consistent with the BSM 
spend plan. The BSM spend plan is submitted as part of the budget formulation process. 
Currently, the account is not apportioned by project in category B, but IRS notifies OMB of any 
deviations from the spend plan that exceed $ 1 million or 1 0 percent of a project’s cost. 

The Department of Veterans Affairs’ (VA) Information Technology Systems account has 
apportioned funding by category B for specific projects in recent supplemental appropriations 
bills. Examples include funding provided for information technology support and improvements 
for processing of Operation Enduring Freedom and Operation Iraqi Freedom veterans benefits 
claims. VA submitted an expenditure plan for category B funds for OMB approval prior to the 
apportionment of funds, which is then reviewed throughout the budget execution process to 
ensure that funds are obligated as planned. 


c. Lastly, do you believe OMB is providing Congress the information necessary to hold those 
agencies in the spotlight that are doing exceptionally well? 

Response: 

OMB highlights agency successes in a number of ways to various audiences and Congress, 
including reports such as the annual E-Govemment report and Clinger-Cohen annual report 
(Chapter 9 of the Analytical Perspectives of the President’s Budget) to Congress. 

Question 3: 

3. Many investments that agencies are undertaking have to deal with their financial management 
systems. As agencies are bringing on new investments, they have to phase out older, legacy 
systems. However, I am little worried that we are investing in new IT systems that may not be 
achieving the best value for the American taxpayers. Specifically, I have a few questions about 
the Department of Homeland Security’s Transformation and Systems Consolidations investment. 
Previous hearings have focused on some of the problems that DHS faced a few years ago when 
consolidating the many stove-piped financial management systems. 

a. Specifically, how is DHS using proven capabilities of commercial-off-the-shelf software and 
in what way is this a factor when selecting a vendor? 


4 



132 


Response: 

All agencies are required to use a COTS software package certified by Financial Systems 
Integration Office (FSIO) as stated under OMB Circular A- 127, Financial Management Systems. 
DHS is not exempt from this requirement and therefore is required to procure a certified COTS 
package. 

b. Further how is DHS making sure the government is receiving the best value for its money 
when selecting IT vendors and system integrators? What specific measures is DHS using to 
measure success for products delivered by IT vendors and system integrators? 

Response: 

DHS plans to follow OMB’s Competition Framework for Financial Management Lines of 
Business Migrations issued on May 22, 2006. DHS shall select a service provider from either 
federal service providers designated by OMB or from commercial vendors. This ensures the 
agency will obtain bids from a large pool of qualified providers, which results in the most 
competitive bid. After selecting a provider, DHS will monitor provider performance using the 
OMB guidance on earned value management (EVM), see OMB Memorandum M-05-23, 
Improving Information Technology (IT) Project Planning and Execution. DHS’s EVM System 
shall be compliant with the guidelines under ANSI/EIA STD-748. 

c. Also what has DHS done to make sure the financial management system consolidation doesn’t 
experience the same problems that were faced in the previous attempt? What is the most 
optimistic and pessimistic cost and delivery date for DHS to consolidate all of their financial 
management systems? 

Response: 

DHS has learned from past experience. For instance, DHS will not rebuild requirements for its 
new system, Transformation and Systems Consolidation (TASC) from scratch as it did with its 
previous attempt. Electronically Managing Enterprise Resources for Government Effectiveness 
and Efficiency (eMerge2). The new acquisition strategy is not a build strategy, but rather the 
adoption of an existing federal capability. A Request for Information (RFI) was released to 
industry in May, 2008. Twenty industry responses were received and ten meetings held with 
industry with DHS component representation attending. The RFP will be released on September 
30, 2008. The award date will be in Q2 2009. In addition, DHS will rely on the standard 
business processes developed by FSIO. The Independent Government Cost Estimate (IGCE) is 
not currently available for public release as the Department is in an acquisition sensitive 
timeframe. The Department can provide this information once the bids are received. 

d. I understand that there are several IT vendors that provide financial system solutions for DHS. 
What are the benefits for using these multiple vendors in the short and long term? And is there a 
benefit for having multiple vendors in comparison to just one? 

Response: 
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We understand that DHS is moving forward with a fuli-and -open competition and the final 
award will be made to a single vendor. The full and open competition will give DHS an 
opportunity to maximize value by considering alternative solutions and selecting the best 
available federal or commercial provider or providers. 

e. Lastly, how is DHS leveraging the current financial management systems that have already 
received a clean audit opinion? 

Response: 

DHS has experienced some success with its components, such as CBP, which received a clean 
audit opinion. DHS will consider current CBP standard business processes as they move 
forward with the systems consolidation effort. The sound financial management practices 
developed in CBP could be extended to other agencies within DHS. 
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